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IF at the Atari Lab 


The Atari Sunnyvale Research Laboratory was founded by Dr. Alan Kay in early 
1982. The charter of the Systems Research Group was to do research about 
and develop prototypes of entertainment products and environments for the 
1990's. The lab was extremely well-funded and staffed with many of the 
brightest and best of the Atari community, recent graduates of the Media 
Laboratory at MIT, and other fledgeling wizards. 


This collection of papers documents my early research in interactive fantasy 
systems. Important contributions were made by several of the members of the 
Atari Systems Research Group. Eric Hulteen designed and implemented a 
media room environment and collaborated on much of the theoretical work. 
Members of the research staff, especially Michael Naimark, Susan Brennan, 
Scott Fisher, Chris Crawford, Alan Kay, Dr. Arthur T. Fischell, and Jim Dunion 
were instrumental in the development of the interactive fantasy concept. 
Outside the lab, other key contributors were Jeffrey Schwamberger, Clyde 
Grossman, Professor Donald R. Glancy of Ohio State University, Doug Lenat of 
Stanford University, and Ray Bradbury. My deepest thanks to all of them for 
their wild brainstorms, thoughtful analyses, and enduring optimism. 


These papers are copyrighted by me because they were never officially 
published or copyrighted by Atari. They should be cited as unpublished papers. 
They are arranged chronologically and most have titles. Many had Atari 
Research Memo numbers at one time, but the numbering system has not 
survived the last several years. Luckily, the author has. 


Readers should know that these papers represent preliminary work. The bulk of 
ork i area was Created after the lab's demise and is published in my 
n, "Toward the Design of a Computer-Based interactive 
‘The Ohio State University, 1986. The dissertation contains a 
if of the theory of IF, a survey of component technologies, and a 
: stem architecture. It also contains a bibliography of resource 
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FIRST- PERSON SIMULATIONS AS LEARNING ENVIRONMENTS 
INTRODUCTION 


I would like to explore and expand upon the recommendation 
found in section 1.2 of An Agenda for Action formulated by 
the National Council of Teachers of Mathematics: ". .. as 
new technology makes it possible, problems’ should be 
presented in more natural settings or in simulations of 
realistic conditions. ” The Council has expressed the 
awareness that the adults of tomorrow must be equipped to 
solve problems and make choices in "high-technology" 
occupations, and that simulations are effective tools for 
developing the necessary problem-solving skills. I want to 
explore the strengths of interactive simulations as learning 
environments, analyze the particular strengths of first- 
person simulations, and discuss the powers of the form and 
their real-world consequences. Ultimately, I want to 
identify the ethical potential of simulations and examine 
its implications as part of the problem-solving skills that 
today's student will bring to bear on the real world. 


INTRINSIC MOTIVATION 


In his thesis, "What Makes Things Fun to Learn? A Study of 
Intrinsically Motivating Computer Games" (XEROX Cognitive 
and Instructional Sciences Series CIS-7, 1980), Thomas 
W. Malone supports the thory that the quality of intrinsic 
motivation optimizes the effectiveness of learning 
environments and activities. Among the key characteristics 
of intrinsic motivation identified by Malone are challenge, 
fantasy, and curiosity. In other words, an intrinsically 
activity engages the user on those levels. 







sattegno and others have suggested that learning 
hanced by providing rich sensory involvement with 
na activity -- visual, auditory, tactile, and 
ines If all of the above elements are combined, 
they i the hypothesis that learning is optimized in 
the presence of qualities which engage the whole person in 
the learning activity. Stated thus, the theory is at least 
as old as Aristotle and has been argued successfully for 
centuries. 


It is easy to see how an interactive computer simulation can 
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incor pOrate most of the above characteristics, including the 
elements of sensory stimulation, multiple levels of 
difficulty, and stimulants of curiosity such as novelty and 
complexity. Excellent models already exist in the category 
of games. The element of fantasy in computer simulations, 
however, hasS powers and applications beyond those usually 
recognized by software designers. Fantasy is seen by Singer 
(19xx) as a means of susstaining a high level of mental 
arousal or involvement, and by Arnold (1979) as a means of 
suspending a child's learned assessments of the nature of 
possibility and thus of his own limitations. 


FANTASY, IMITATION, AND CHARACTER 


The presence of fantasy in an educational computer 
simulation would seem to violate the NCTM recommendation 
that "problems should be presented... in simulations of 
realistic conditions." But unless the computer simulates 
exactly what a child of eight would do in the absence of a 
computer, the element of fantasy is already involved. The 
real question is how does fantasy work in a learning 
activity, and how can its powers be employed to teach the 
problem-solving skills needed in the "real world?* 


Aristotle observed in the Poetics that humans learn by 
imitation. To imitate is to engage in first-person fantasy: 
"What if I were a flower, a cowboy, a mommy, or a starship 
captain?" It would seem to follow, then, that the most 
educationally powerful simulations are those in which the 
user has a first-person role; that is, the uSer actS as a 
character in the action of the simulation. But how does the 
simulation shape the first-person fantasy, or character, of 
the user? 


In answering this question, the similarities between a 
simu and a drama are instructive. A simulation, like 









a 8B an imitation of an action made up of a series of 
inci Characters move the action forward by making 
choi. and acting upon them. Ina play, the form that the 
charaet@r's actions will take is defined by the form of the 


plot -- the series of incidents that make up the whole 
action. The material of “stuff" of which the character's 
actions are made is the matrix of words and thoughts offered 
up to him by the world of the play. The actor, however, may 
not interject his own skills or judgments into the 
character; the imitation is "“closed" by the script. A 
simulation is very much like a play, except that the "stuff" 
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available to the user for formation of his character 
includes not only the materials offered up by the world of 
the simulation, but also his own knowledge, skills, and 
faculties. The events that occur on the basis of his 
choices and actions are not fixed in a script, but flow from 
the interaction. In this sense, the simulation is an “open” 
form. The user has a double matrix of possibilities for 
action. 


CHARACTER AND CHOICE 


The ability to climb into a role or character in a simulated 
environment may “engage the whole person" in one imaginative 
stroke, presenting at once a new world of conceptual and 
sensory materials and a rich matrix of new possibilities. 
The applicability to real-world problem-solving seems clear: 
simulations can provide opportunities to develop the skills 
needed to assess new situations and to conceive of new 
problem-solving strategies. What may not be clear are some 
of the other levels of learning that are accessed by 
first-person simulations and the strengths and dangers 
inherent in them. 


A great deal of work has been done to extend the levels of 
cognition accessed in learning activities “downward” from 
the computational and rational to include the intuitive and 
sensory. What is: not acknowledged is that as we move from 
the abstract to the concrete, from columns of figures to 
contextual problems involving whole actions and whole 
people, we are also engaging the upper levels of analysis 
and choice-making in a world charged with meaning. 


In the protean game of “Hammurabi,” the user plays the role 
of king with the task of controlling resources in order to 

£¥<;-the populace and maintain his position of power. 
t exercise particular mathematical skills and 
is fairly obvious. In some versions of the game, 
mn be encouraged to access the compter as a _ tool 
complex calculations, thus learning something of 
| f the computer in problem-solving activities. But 
what he is also learning by default in all versions of the 
game that I have seen is that there are acceptable levels of 
death by starvation which must be incorporated in the 
"winning" strategy. 









Characters make ethical as well as expedient choices. In 
the imaginative leap that engages the whole person of the 
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user, tt ethical category is opened, and what may fill it 
is something that the designers of the game neither 
perceived nor took responsibility for. 


Should some other ethic have been wired into the "Hammurabi" 
simulation? Must teachers of mathematics now assume 
responsibility for identify appropriate ethics for their 
students? What happens when a "good" ethic is wired in? 


In the game of "Star Raiders,” the user must blow up as many 
enemy ships as he can in order to protect the galaxy. There 
is no “negotiation” option. There is no interaction on the 
ethical level. The user's choices are binary: either blow 
up the enemy ships and win, or don't and lose (or be 
perverse and blow up your own starbases). Although the 
ethical stance ("protect the galaxy") may be deemed 
acceptable, the ethical faculty has not been exercised or 
developed; it has simply been defined by default. 


Are such scenarios really conducive to the development of 
problem-solving abilities that are applicable to the real 
world? How will they serve the genetic engineer, the 
nuclear physicist, or the concerned citizen of tomorrow? 


FACING THE ETHICAL COMPONENT OF SIMULATIONS 


The ethical component of first-person simulations is real. 
As an element of intrinsic motivation, it is powerful. In 
the skills matrix of real-world problem solving, effective 
use of the ethical mode is enormously important, both in the 
process of formulating key questions, and in the ability to 
visualize and evaluate consequences. The question for 
software designers and teachers is not whether the ethical 
dimension exists, but how the ethical faculty can be 
and developed to the end of effective 
ring in the real world. 








golution is flexibility: educational 
should allow the user to play with different 
‘and to observe their consequences, or to 


‘@ explicitly the ethical dimensions of the problem. 
Programs must be designed to recognize user strategies and 
demonstrate their implications. The path of possibility 
from choices on any level must be demonstrated in the 
first-person mode. Finally, the ethical component should be 
opened up for user interaction and discovery. 
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The first and most important action we can take is to become 
aware= of the ethical dimension inherent in the form of 
first-person simulation. We must observe how it is defined, 
who defines it, and what -- whether we like it or not -- is 
ultimately being learned. 








RESEARCH PROPOSAL: THE POETICS OF INTERACTIVE FORM 
re NE NTERACTIVE FORM 


The objective of the research is to analyze the powers of certain 
interactive works as art. We sense intuitively that interactive 
works can have art-like powers--the ability to arouse emotions 

Or to stimulate a sense of beauty, for instance. The goal is 

to identify those powers, and then to discover the formal and 
structural mechanisms through which they are realized. 


The similarities between certain interactive works and dramas are 
striking, and they Suggest that the existing body of dramatic 
criticism can provide a place to begin the work of analysis. We 
can identify a class of interactive works that are Mimetic; that 
is, they imitate actions, things, and events that do or might 
occur in some real world (the worlds of Oedipus, Hamlet, and 
Frodo Baggins are equally "real" in the sense of dramatic 
probability). 2 


‘We can identify a sub-category of mimetic works whose primary 
intentions involve entertaining, engaging, or pleasing the 
humans who play with them, thereby excluding works whose goals 
are exclusively computational, analytical, or even instructional. 
The resulting class may be called poetic, and it includes many 
games, computer-generated worlds, simulations, and a whole host 
of unborn genres. 


Within such poetic works, we can identify elements that correspond 
to the primary elements of the drama: 


SPECTACLE: everything you see 

MUSIC: everything you hear 

DICTION: © meaning-bearing signs and symbols 

THOUGHT: mental processes implied by what is 
Said and done 

CHARACTER: a bundle of traits or predispositions 
to act in a certain way 

PLOT: the whole action being imitated 

5. 


Likewisey:we can investigate interactive works in terms of some 
traditional aesthetic criteria, such as: 


MAGNITUDE: of a size that can be taken in by the 
viewer, so that it may be perceived as 
beautiful | 

RHYTHM: the pleasing arrangement of action over 
time 

WHOLENESS: organic unity 


(next page) 
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The straightforward application of such critical concepts can 
help us assess the quality of interactive works in ways that are 
already different from the standards that are normally applied 
("human factors" criteria, focus group results, product test 
Criteria,  éte,) « 


The most interesting part of the research, however, requires the 
formulation of new theories in areas where linear dramatic and 
interactive forms diverge: the realm of the user as an agent 

in the dramatic action. Our understanding of the drama depends 
upon the plot as the whole action and formal cause of the piece. 
By formal cause, we mean that which provides the organizing 
principle for all of the other elements of the piece. But what 
happens when the shape of the whole action is variable, depending 
upon the real-time choices of an intelligent agent who exists 

at once inside and outside the work? Is it possible to design 

a scheme of probability, a frame for user action, that can 

still provide the necessary organizing principle? Or will the 
intrusion of that willful agent blow the form apart? What we 
are seeking is a structural understanding of beauty in a made 
thing that accomodates the unpredictable. 


The preliminary work on this question is theoretical. By 
playing with the dynamics of formal. and material causation, we 
should be able to arrive at a theoretical model. That model. 
might take the form of a mutable character, defined in terms of 
both dramatic probability and degrees of freedom. From there 
we should be able to predict the structural effects of mutable 
character on the piece. The next step will be to create a 
"living" software model that we can observe and tweak. The 
criteria for success will be aesthetic in nature: questions of 
magnitude, wholeness, and other principles of beauty can be 
applied. Of course, we will want to ask the obvious: "Does 
the piece work as art in every instance? Is it emotionally 
Satisfying? Organic? Pleasing? Beautiful?" 





For the known and potential uses of our medium, the implications 
are quite. powerful. Perhaps we can learn how to invoke the 
powers Of @mpathy, emotional arousal, and catharsis in such 
"open-en@@d” works. Can we find a real-time aesthetic that will 
transforiour hybrid efforts into a new way of making art? Can 
we employ computer technology to exponentiate artistic pleasure 
and richness as it has informational and computational power? 






I believe that we can. The form has tremendous powers that we 
have yet to actualize, and that actualization depends as much 
upon new artistic theory as upon new technology. The idea of 
mutable character is but one facet of our new "poetics." The 
introduction of that or other equally powerful concepts may 
allow us to systematically transform the poetics of linear 
drama into a comprehensive theory of interactive works as art. 
That is my hope. 


(next page) 
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What form can we expect the proof of such a theory to take? 

A single exemplary work could be "merely" serendipitous. One 
thing we can do is aoply our new theory, in the form of 
CYititcism,. to existing works and evaluate the effectiveness 

of subsequent transformations. Numerous candidates exist! 

A more generative application of theory lies in the design of 
authoring environments. To some extent, every work reflects 
the assumptions inherent in the tools used to build it. Most 
computer programs reflect the logical/mathematical grammar of 
their authoring languages; Many shortcomings in human factors 
are attributable to this bias. With an art-based theory of 

the nature of interactive works, we should be able to construct 
an authoring environment that emphasizes dramatic and aesthetic 
principles over logical/mathematical ones. Such an environment 
would open the medium as a Creative palette for non-engineering- 
Oriented artists. An interesting proof of our theory would be 
a work, employing the full power of the medium, created by 

Ray Bradbury, Arthur Miller, or Frank Frazetta with such 
authoring tools... 

The user-participatory possibilities inherent in interactive 

form appear to obscure its status as art. When the line between 
artist and audience/user becomes blurred, things tend to collapse 
in chaos. The artist, sensing loss of control, may simply walk 
away from the form as a dead-end experiment. The haphazard 
interaction of audience members in sixties' theatrical experiments 
like "happenings" and some performances of the Living Theatre 
created just such un-beautiful, inartistic experiences; both 
genres have disappeared. But when the energies of artist and 
audience/user are woven, as thesis and antithesis, into a new 
artistic whole, the possibility exists for an artistic synthesis 
in which real formal evolution can occur. Jerzy Grotowski's Poor 
Theatre, for instance, casts audience members in choral roles, 
providing a means for dramatic integration of their participation 
which enhances rather than erodes the artistic power of the new 
theatrical form. 


We brush against these questions every day in our work in human 
factors, game design, and even artificial intelligence, but 

we rarely: conceptualize them as artistic questions. The 
tremendows body of knowledge in literary and theatrical criticism, 
as well as theories drawn from linguistics and structuralism, 

can surely inform our efforts to find "what works." 


B. Laurel 
12/10/82 





INTERACTIVITY 


Conversation with Michael Naimark, 4/4/83 


IMPLICIT VS. EXPLICIT INTERACTIVITY: Naimark sees this as a 
major distinction, and presented the following examples: 


A. A user may select one of thirty video "channels" (offerings, 
pieces) by pressing one of thirty buttons. The channel of his 
choice is then displayed on the (single) monitor. 


BR. A user may select one of thirty video channels by deciding 
which of thirty video monitors, each displaying a different 
channel, that he wishes to attend to. 


Case (A) is "expliciti" case (B) is "implicit." &@ goal would 
seem to be to strive for systems that operate with implicit 
interactivity; that is, the user is never asked to step back and 
make an explicit choice. 


SIMULTANEITY 


How might implicit interactivity be expressed? Naimark noted 
that Andy Lippman, current head of MIT AI lab, believes that the 
quality of "Simultaneity"” is criterional of true interaction. By 
this he means that the user and the system must be simultaneously 
monitoring one another. The interaction is an ongoing 
collaboration, with "inputs" (lousy word) from both parties (user 
and system). 


This is related to the idea of a system that "models" the user in 
some way and implements implicit (potential?) choices by noticing 
haw the user is behaving in relation to the system--his learning 
or browsing styles, for instance, or his apparent skill level (in 
the case of a game ocr CAI activity). This scenario is a bit 
insidious. 


OBSERVAT PONS 


I think that the implicit/explicit distinction is a useful one, 
but insistence on implicit interactivity as Naimark defines it 
might preclude some forms of overt cheice-making that could be 
pleasurable. The distinction can be made finer in the context of 
interactive drama. 


To have the experience of behaving like a character inside an 
imaginary world, the user must be allowed to make explicit 
choices in the PERSONA of the character. Assuming a WILLING 
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SUSPENSION OF DISBELIEF, then those choices are made in the first 
person. The operarid(s) of those choices should always be within 
the imaginary world itself; that is, the user should not be 
required to have explicit interaction with the SYSTEM or to make 
choices at the system level. 


Rather than defining choices made by the user "in character" as a 
subset of what we have called explicit interaction, it would be 
more productive to posit the existence of a third category of 
interaction: DRAMATIC INTERACTION. We will use the term 
"explicit interaction” to refer to those choices made at the 
system level, and not within the context of the imaginary world 
that the system imitates. We will use the term “implicit 
interaction," as we have defined it above, to refer to those 
cases where the user’ss influence on the system (and thus om the 
experience itself) is accomplished indirectly, without the overt 
expression of choice. An example of implicit interaction in the 
context of interactive drama would be the modification and 
subsequent impact of probability heuristics within the system. 


(SEE color diagram, "KINDS OF INTERACTIVITY", 4/6/83) 


The most important disitinction among the three can be expressed 
as the EXPERIENCE OF PERSON-NESS. We wish to preclude “explicit 
interaction" because we sense that it is intrusive--that it 
interrupts or even destroys the user’s ongoing exerience by 
forcing him to shift his attention to another order of problems. 
This is why branching interactive movies, videodiscs, or software 
packages feel like a kludge. Once the user is "inside" the 
interaction, we want him toa stay there and be perfectly 
comfortable; in other words, we want him to be having a "first- 
person" experience. 


In dramatic interaction, then, the user stands in first person 
relation to the experience. In explicit interaction, the user 
operates in second person relation to the system, making, as it 
were, eee arte statements to it. In implicit interaction, the 
imself to be in third person relation to the system, 
invited ‘or able, unless he is very clever) to 
control the way he is impacting the experience. 








THE MIDDLES ARE BORING 


Conversations with Naimark, #2 
4/13/83 


"The middiles are boring" is a direct quote from Dr. Alan Kay, and 
an observation that helds true in a wide variety of contexts. In 
an "interactive branching" scenario, whether in computer or 
videodisc software, the response of the system that directly 
follows the user’s choice/action at a "node" provides the user 
with a "first-person rush"--an immediate sensation of 
interactivity or power. Lipmann observed the phenomenon of this 
"rush" at the branch points in the Aspen Disc: when a user 
directs the system to "turn left," it is the actual turn-~the 
transition which immediately follows the choice--that is such a 
Pleasurable moment. The linear sequence which follows the choice 
nodes that is, the journey through the movie-map to the next 
corner or node were choice is possible, is comparatively boring. 


In most current interactive computer software, the same 
phenomenon occurs. The location of the choice "node" is, if not 
explicit, at least quite poorly camouflaged and readily apparent 
to the user. In the case of most current models of interactive 
fiction, for instance (the text adventure game is a good 
example), the user knows that once he has made his input, he must 
now sit back and view the results. In all but the best of 
systems, those results are also quite obviously predetermined, 
implying that the user choice has been forcibly interpreted as 
one of a finite set of anticipated responses. 


The net effect may be expressed as a disruption in the 
experienced person-ness of the interaction (SEE interact.x for a 
discussion of kinds of interactivity). Following the choice node, 
the user passes quickly from a first~ or second-person experience 
to a third-person one. 







At this | 
third-pa 


it may be worthwhile to try to describe how a 
@eperience feels to the user. Moving back to the 
grammatii ociel for a moment, we notice that by “third person" 
we mean » prenmouns, he, she, it, and they. Now, in every 
"interactive" experience the assumption is that the user is a 
participant. In a third-person experience, however, there is no 
"I" to feel or act; there is not even a "you" that might dencte 
the system’s recognition of my presence. Either I become 
invisible to the system in that it is ignoring me completely, or 
I am suddenly dragged through the portal-—--through the proscenium 
arch, as it were--to be one of the many he-she-it entities on the 
other side. 


It is no wonder, given the existential implications of this 
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predicament, that sudden shifts to the third person can cause a 
certain malaise. The obvious aesthetic consequences of such a 

disruption are but a superficial subset of the cognitive ones: 

the user is literally yanked into a view in which he completely 
disappears. 


(NEXT: How videcgames succeed. ) 
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WILD SPECULATIONS: 


Interspecies Game 


Is there a video game we could imagine that a human and a dolphin 
could play together? Of course, HCD couldn't sell it (without an 
extensive marketing effort to get hardware into dolphin homes, or 
into all those joints where humans and doiphins hang out together). 
However, such a game would let us ask some real basic questions 
about the nature of interaction and "fun." We could also ask some 
esoteric ones about the nature of imitated action, semiotic thres- 
holds, cognitive and kinesthetic response patterns, etc. etc. Ail 
this in addition to the obvious questions about interspecies comnu- 
nication! 


spinoff Game 


Successful design/implementation of the above would give us enough 
data about dolphins to simulate a dolphin persona inside a video 
game for humans, yielding a superb learning/game product, based on 
a topic with tremendous popular appeal. 


Thoughts 


Seriously now, folks, this is just crazy enough to be great. I suggest 
that Sandra and I and whoever else is interested go brainstorm with 

the Lilly folks and come up with two proposals: one for research, and 
one for @ consumer product. Whaddaya think? 
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The "Tool! Metaphor 


Some Speewul ations 
i > 

EB. Laurel: 

6/20/83 


The idea of computers (and/or software) as tools has been the 
most pervasive metaphor applied to the technology from its 
inception. I want to start by suggesting that "tool" as a idea 
does not describe the whole potential of computing powers there 
are kinds of computer uses and experiences that are possible 
which are not at all tool-like. In fact, the tool metaphor is 
often destrictive because it induces us to build inappropriate 
and dysfunctional interfaces. 


Let’s start by thinking about what we mean by "tools." I can 
think of three varieties right off the bat, and I’m sure there 
are more, distinguished by their functions and uses. 


The most common sort of tool (at least until quite recently) is 


manipulate real-world objects. The desired outcome of its use is 
also an object or event in the real world. The function of a. 
hammer, for instance, is to drive or extract nails; the desired 
outcome of its use is to construct or dismantle real-world 
objects. The function of a pen is to apply ink to a surface in a 
controlled way; the desired outcome is to create text or 
drawings. 


function is to augment our sensory capabilities. The outcome of 
its use, however, may or may not be a real-world object or event. 
A microscope, for example, extends our visual perception. It may 
be used to learn more about the nature of cells ("pure" science) 
or to isolate a disease agent in order to affect a cure 


("applied" science). 


A third kind of tool can be described as a symbolic tool. It 
uiates nor amplifies real-world phenomena, but 
‘® on symbols. Like a perceptual tool, the intended 


wee may be "pure" or "applied." The calculator is 







tools, the function is distinct from the outcome. One rarely 
uses a hammer for the sheer joy of randomly driving nails. If 
that were the intended use of a hammer, we would probably 


describe it as a toy rather than a tool. 


The difference between a toy and a tool is just this: for a toy, 
there 15 no distinction between function (what it DOES) and 
outcome (what we want to accomplish by doing it). The experience 
of the toy’s function is pleasurable in real time. The function 





IS the outcome. The same thing may be said of art. 


Computers and software can indeed be used as tools. A word 
processor, for example, has the function of creating and editing 
text, and the sutcome of producing documents, files, Code, etc. 
A calculation program does not significantly differ from a 
calculator or an abacus in terms of function and outcome. In 
enhanced versions of other kinds of tools. Although such tool- 
like uses of computers may be functionally more complex and 
efficient and may produce outcomes that are much greater in 
magnitude than those of the tools they emulate, the basic natures 
of their functions and outcomes remain the same, as does the 
distinction between them. 


Such applications are, as we know, a subset of what computers can 
be made to do. Some applications, like adventure games, we might 
describe as "toys." Others, like synthesized music, could be 
described as "art." If we wish to persist in defining the 
computer itself as a tool, then the associated functions and 
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outcomes must encompass all such uses (more of this later). 


How, then, is the tool metaphor destructive? Let’s consider an 
example of a computer activity: the garden variety adventure 
game. The function of the program is to allow the user to "play" 
im an imaginary world. She moves through unknown spaces, usually 
with the goal of gathering treasures, while avoiding or 
overcoming obstacles and gqnarly monsters. The outcome of the 
experience is the experience itself -—- the pleasure of the 
activity in real time (expertise gained through repeated use 
serves to enhance this pleasure). 


But let’s look at what the interface is telling us about the 
functional qualities of the program. The user may press "N" to 
move north, for example, or “{control)lobj" to view a list of 
objects. If she wants to change the qualities of her magical 
sword, she may suspend play, view and manipulate a list of 
attributes, then re-enter the action of the game. In this view, 
the interface functions to allow the user to communicate with the 
program, as: distinct from pursuing that "pleasurable experience" 
wh oh MR 

POO) shi 
where, 
same, 
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have observed, they are theoretically one in the 


This inappropriate distinction is an artifact of the tool 
metaphor. If we think of something as a tool, we assume its 
function and outcome to be distinct. The tool-like “access 
language” of the interface is pulled into being by the force of 
the metaphor. The result is a kind of estrangement, a distancing 
from the very experience that is the whole raison d’@tre of the 
program. This habit of mind, more than hackerliness or the 
"state of the art,” explains the common weaknesses of the 
human/computer interface. 
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To return.to an earlier question, if we try to think of the 
computer itself as a tool, how do we define its functions and 
outcomes? I suggest that its function is to represent a wide 
array Of phenomena, including tool-like and un-tool~like. 
activities. The outcome of its use, it seems to me, is the 
representation iself. This is the meta-level that I think we 
fail to grasp. Even when we use the tool metaphor appropriately 
in particular applications or activities, we invoke it 
inappropriately in our image of the technology itself. An 
example of this phenomenon is a piece of file management software 
being worked on by some of the people Kristina and I encountered 
at UCSD last week. The "interface problem" was described by one 
of the programmers thus: "How do we make it easy for the 
secretary to know what "\\’del" is good for?" Thus, even ina 
tool-like activity, functions are split from outcomes at the 
whole-system level so that the "access language" does not manage 
files, but rather talks to the file-management program! 


The tool metaphor at the highest level dooms us to the recursive 
nihtmare that plagues every discussion of human/computer 
interface. 


(Next flame: New Metaphors) 








Exerpts from Two Interactions 
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NOTE: The text in UPPER CASE represents user input. Gestures, 
visual output, etc. described in the "stage directions" in 
parentheses. 


(Looking out forward view screen at long-range view of planet) 
MR. SPOCK’? 


SPOCK: Unfamiliar technology and code, captain, but the distress 
signal appears to have originated here (viewscreen shows 
magnification of equitorial region, with blinking pointer). 


FALL. INTO STANDARD ORBIT, MR. SULU. LIEUTENANT UHURA, KEEP 
SENDING THAT REPLY AND LET ME KNOW THE MINUTE YOU ESTABLISH 
CONTACT. MR. SPOCK, DR. McCCOY, MEET ME IN THE TRANSPORTER ROOM. 
(Turns to Intercom) SECURITY -~-? 


REILLY: Yes, Captain? 


TWO ARMED SECURITY PERSONNEL TQ BEAM DOWN WITH US —- IN FIVE 
MINUTES. 


REILLYS Yes sir. 


(Flips off Intercom, returns to first-person visual of the 
bridges moves to lifts’ sound of doors closing behind) TRANSPORTER 
ROOM. (Sound of lift engaging) 


IF THAT @E@TRESS CALL IS GENUINE, WE ARE GOING TO HAVE TO 
INTERVENE IN THEIR CIVIL WAR TO AVOID THE COMPLETE DESTRUCTION OF 
ONE OF TH@®-SENTIENT SPECIES ON THE PLANET. 






"I don’t have enough information to consider violating the prime 
directive -- better not pursue this line of thought until I get 
down there and see for myself." 


RIGHT. (Lift deers opens visual scan of corridors walks to 
transporter room. SPOCK, McCOY, and SECURITY GUARDS are already 
in place to beam down. Walks to transporter platform). 
ENERGIZE. .. 
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BUILD A WORLD, PLEASE... 
"Right. I’m listening." 


USE WHAT YOU KNOW SO FAR ABOUT MY FAMILY AND GIVE ME WHAT YOU CAN 
PUT TOGETHER ON 1966. IN THE KITCHEN, EVENING, MOM AND DAD AND 
ME SITTING AROUND THE TABLE. 


"Right. I have personality information on your parents and I 
know something about what that kitchen looked like. How’s this?" 


GOOD. MAKE THE TABLE A LITTLE BIGGER, AND MAPLE, NOT WALNUT 
- THE WINDOWS ARE RIGHT BUT THE CURTAINS SHOULD BE ORANGE, AND 
CLOSED . . . SHE ALWAYS KEEPS THEM CLOSED. 


"T° ll remember that." (Screen image is adjusted) 

HERE’S THE SITUATION. JIM AND I HAVE BEEN OUT IN MY CAR LAST 
NIGHT. THIS MORNING MOM FOUND HIS WALLET AND COMB IN THE BACK 
SEAT s s a 


"Excuse me, can you tell me something about Jim?" 


OH YES . . . JIM MARTZ .. . YOU SHOULD BE ABLE TO FIND HIM IN 
THE DIARY . . . HE WAS MY STEADY BOYFRIEND SINCE *64... 
"Right. Goon... " 


SQ MOTHER FINDS THIS STUFF AND DECIDES THAT WE ARE SCREWING. SHE 
DECIDES TO FORCE US TO GET MARRIED. HERE’S THE DIALOGUE AS I 
REMEMBER IT... . 





Yes. you want to do? 


DKAY, PLAY ME WHAT YOU HAVE AND I WANT TO JUMP IN ABOUT THE TIME 
SHE SAYS "WE WILL GIVE YOU YOUR BEDROOM FURNITURE AND TWO 
THOUSAND DOLLARS ..." 

Okay. Ready? 


(Dialog begins, goes on for a few minutes) 


MOTHER: . . . and we will give you your bedroom furniture and two 
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thousand dollars a a 


WAIT A MINUTE, MOM. I DON’T THINK IT HAS TO WORK OUT THIS WAY. 
WHAT WOULD IT TAKE FOR YOU TO FEEL BETTER ABOUT THIS... ? 
(At this point the dialog begins to diverge from the original 
scenario . . .) 








2 


4 


es ——<— se ae mS ee ete — << 
— ee ee ee ee ee ee ee ————— — 


— me oe —— — ee ee ee oe ——— = oe Se 


B. Laurel 
Created 8/11/83 
Last Revised 8/12/83 


Overview 
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The idea of attaching an intelligent playwriting system to an 
= story generator is attractive for several reasons. The 
o¢ problems of representing characters and providing the means for 
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them to make choices that are manifest in action are largely 
solved, at least conceptually, in a system like Meehan’s Tale- 
Spin. Further, the character and thought levels of Tale-Spin are 
already equipped to listen to the advice of the "Storyteller" 
portion of the program, which currently functions, in one mode, 
to apply story-based criteria to clip redundant, unimportant, or 
bering detail from the output story and, in another mode, to 
synthesize a story to exemplify a fable or moral. The next 
logical step is to replace (or transform utterly) the Storyteller 
with a program which aims to alter the output of the system in 
two directions: to enable a user-character interface, and to 
employ dramatic techniques to augment "interestingness." 


To implement some important dramatic techniques, it will also be 
necessary to expand the scope of human-authored materials upon 
which the system may operate, specifically to provide materials 
which are generative of accidental incidents and of dramatic 
reversals. Although we may wish later to develop ways to 
automate the authoring of materials (technially, the 
specification and delimiting of POTENTIAL), assuming that such 
authoring will be "hand-done" for the moment will allow us to 
focus upon the design and implementation of a playwriting expert 


as our first order of business. 
ee: 





Tale-Spin deals with the generation and telling of stories and, 
as such, uses a narrative form: the stories are episodic in 
construction, and are meant to be read by a non-interacting user. 
Once the second-person transactions with the system that are 
necessary to initiate action have been completed, the user’s 
experience of Tale-Spin is essentially in the third-person mode. 
It is important to recognize also that the user need only 
establish the initial canditions af the story; the system 
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operates independently to synthesize the story from these inputs. 


The interactive system which we envision is different in several 
crucial aspects. We wish to enable the user to move about in the 
"story" as one of its characters. First and most important, this 
would transform the user’s experience inta a first-person mode. 
The operation of the story generator itself would also be 
fundamentally changed. Currently, Tale-Spin allows the user to 
describe characters with traits and goals. The system then 
Creates those characters and those features of an environment 
which are necessary to them, AND NOTHING ELSE. The story is 
generated from these materials, using the planning and action 
structures that are part of the “generator” itself. 


Allowing the user to move about in the story as a character means 
that the user-character’s choices and actions must be taken into 
account at every moment, as they are continually affecting the 
states, goals, and actions of other characters and the state of 
the "world" at large. What must the system do to accomodate this 
stream of inputs”? 


First, it must provide a means for the other (generated) 
characters to form models of the user-character. Each 
character's ED, be based on the information about the 
user-character to which he has been exposed, as well as upon his 
own traits and predispositions (literally, what he is inclined 
and equipped to notice). Second, the system itself 
(specifically, the "playwriting" program, of which more later) 
mist maintain and update a model of the user-character in order 


t 7 to know what is ultimately "true." This is a harder problem than 
Pisa it seems, for without sophisticated user constraints, the system 
is theoretically no better equipped than any of its characters to 


know the actual states and goals of the user-character. 


Further complications arise from the fact that the user-character 
does not have the same organic relationship to the potential 
embodied in the story world as one of the characters that has 
been cre with it. By their very nature, system characters 






are inc of introducing new potential into the story world — 
~ they & nly formulate that potential into possibility in the 
pursuit % heir goals. But what is to keep the user-character 


from introducing new potential (that is, materials which are 
unknown to the system in its current configuration) and thereby 
_~ blowing the system up? The essential user-character problem is 
LW v one of information: how can the story world (system) and the 
wo user-character exchange necessary and reliable information 
without violating the first-person imperative (that is, without 
resorting to second-person techniques like "“question-and-answer" 
and "error messages”)? 


Three distinct solution strategies can be used to address the 
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various aspects of the problem. The problem of communicating 
information to the user-character about the potential of the 
story world ina first-person mode can be re-cast in cognitive 
terms: how does the user know what is possible, or "where the 
edges are?" The concept of EXPOSITION, borrowed from the drama, 
suggests a solution that operates in two ways. In plays, 
"exposition" is the presentation of materials and information 
which define the limits cf the world of the play (context). The 
total of expository materials is made available to the audience, 
50 in a way, the audience has the same kind of global access to 
context information as the playwriting program would have in our 
system. Exposition is also relevant to the specific characters 
in the play. Many of the interesting complications in a drama 
result from the differences in content (completeness and 
reliability) as well as order of presentation of expository 
inforation to various characters. This character-specific 
exposition corresponds to the distinctions in knowledge about the 
story worid held by the various characters in our system. 


In a play, exposition is an ongoing process, operating through: 
and around the dramatic action, with a higher concentration of 
exposition occurring at the beginning of the play. Exposition 
would occur similarly in our interactive system, with the highest 
concentration at the beginning of tha ineeearbiens We might 
propose an “expository interval" before the ‘dramatic action 
begins, during which the user would simply observe the "world in 
progress,” gaining information about the context and characters. 
This strategy assumes that the entrance of the user-character 
into the story world functions as a sort of inciting incident, or 
formal beginning, of the dramatic action. Later, expository 
material could be doled out to the user-—character as well as to 
the other system characters, according to access opportunities 
(being in the right place at the right time), goal to know (as 
Meehan’s characters can question one another, for instance), and 
the potential for each morsel of information to create 
interesting dramatic situations (determined by the playwriting 
program). 







Hnigue for creating cognitive boundaries is to 
ho assertions on the part of the user-character are 
acknowleaged by the system and become manifest in the action. If 
the user-character attempts to introduce or operate an potential 
that is beyond the boundaries of the story world, that assertion 
can simply fail to produce results. If user-character Joe Bear 
attempts to solve a problem by asserting that he has magical 
powers that will enable him to find honey, for instance, the 
action subsequent to that assertion can simply prove him wrong -- 
at the same time supplying the information that such powers are 
nat part of the potential of the story world. This technique, 
while obvious and simple, is less intrusive and destructive of 
first-personness than a second—person question or error message. 
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(It also raises the whole issue of the relationship between 
degrees of freedom and the potential for pleasurable experience, 
which is beyond the scope of this paper but will be treated 
later). 


A&A third facet of the user-character problem is haw the system can 
construct a model of the user-character for its playwriting 
operations which has a maximal degree of reliability. Of course, 
the two techniques for informing the user about the potential of 
the story world will help to prevent accidental errors an the 
part of the user in communicating his own goals, traits, and 
strategies, but will do nothing to preclude intentional 
misrepresentation. Little attention can or should be paid to 
intentional misrepresentation as vandalism, since human 
perversity will always find a way to assert itself anyway, but 
misrepresentation for dramatic ends (as part of a strategy for 
achieving a character’s goal, for instance) should be accomodated 
and utilized as elegantly as possible. 


The technique which the playwriting program might use is one of 
Successive approximation, making and updating a “best guess" 
version of the user~-character model whenever the user-character 
acts (action in this sense includes both sayings and doings). 
That best-guess model can then be used to orchestrate the lines 
of dramatic probability in the story world, and less reliable 
versions may also be used to create and maintain alternative 
probability structures (which will in turn enhance both the 
complexity of the plot and the richness of dramatic alternatives 
available to the playwriting program). When certain elements of 
the user-character model are shown to be erroneous, the relevant 
lines of probability can be re-directed on the fly, and the 
alternate lines can be examined for their usefulness in light of 
the new information. How lines of probability are configured 
will be discussed below. 
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Dramatic probability may be defined as the causal relations among 
incidents in a play. In review, we know that a play 1s an 
imitation of an action and that it has a beginning, a middle, and 
an end. While a play is constructed of many actions (and their 
agents), it is ultimately an imitation (or representation) of a 
single, higher-order action which unifies the whole, and it is 
the progression of this superordinate action which provides the 
beginning, middle, and end. Dramatic probability is the means by 
which the action moves forward. 


We can describe all the potential for dramatic action which 
exists at the beginning of a play as a world of POSSIBILITY. 
Possibility, then, includes anything that might or could happen 
within the context of the play’s world. Hamlet at the beginning 
of Hamlet, for instance, could possibly return to school after 
his father’s funeral. Why not? Forward motion in the plot is 
accomlished through the formulation of possibility into 
probability. As Hamlet unfolds and Hamlet has a conversation 
with his father’s ghost, it becomes PROBABLE that he will remain 
in Denmark, at least until he has more information about the 
ghost and the circumstances of his father’s death. Later in the 
play, as Hamlet verifies the ghost’s story in various ways and as 
we see Hamlet’s deep loyalty ta his father and his desire for 
revenge emerging through the pattern of his actions, that line of 
probability is further refined to predict that Hamlet will remain 
in Denmark until his father’s death is avenged. 


Ja 










ya, several major and often contradictory lines of 
‘@eiet through the play until the final moments. It 
ole that Claudius will eliminate Hamlet before he 
father’s death throughout much of the play, but as 
the achic receeds, it is also probable that Hamlet will succeed 
in killing Claudius. (By contrast, it is not probable that 
Ophelia will slay Hamlet in revenge for the death of Polonius, 
judging from her actions and what we infer about her character). 


At the end of a play, one line of PROBABILITY is finally 
formulated into NECESSITY. Again in the case of Hamlet, the 
probability that the evenly matched wills and intentions of 
Hamlet and Claudius will produce a conflict that ends in the 


death of both characters becomes inevitable, and other lines of 
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probability are withdrawn. It is this final movement from 
probability to necessity that signals the end of the action (and 
of the play). From the vantage point of that final necessity, we 
can clearly see the causal chain of events that led to the 
Sutcome, producing the emotional and cognitive sense of closure 
that we feel at "the end" {we have probably all seen films of 
Plays with “crummy endings" which failed to meet the causal, and 
therefore the emotional and cagnitive criteria, of dramatic 
necessity). 


The causal linkages of events, then, established the movement 
from possibility to probability to necessity and provides the 
play with its emotional shape. A story generation program like 
Tale-Spin is capable of creating one type of causal linkage which 
establishes probability and unifies the action of the story: the 
causality of goal-oriented behavior. The central character has a 
goals; there are obstacles to his goal which consist of the goals 
and traits of other characters (nested hierarchically); and the 
character ultimately either succeeds or fails. Here is a simple 
Tale-Spin example: 
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Laurel and Hulteen 
Fischell—-Turing Productions 


Our initial proposal involved a production of an existing script, 
THE VELDT by Ray Hradbury, in the media room. The script was 
chosen because it.features a futuristic children’s playroom which 
is quite similar to the media room we are developing here at ASR. 
The production was to provide motivation and direction for 
"bringing up" certain aspects of the media room and especially to 
Give us the occasion to work with video and animation on the 
walis of the space. We were also interested in using the project 
to attract the attention of Bradbury, in hopes of persuading him 
to collaborate with us on other interactive projects. 


The primary shortcoming of the original project was the lack of 
opportunities for real-time interaction due to the presence of a 
"closed" dramatic script. While the design and realization of 
the media and environment were directly inline with our common 
research goals, the dzrth of interactivity made those efforts 
less generalizable to other projects. The temptation to revise 
the script in order to create a more interactive environment was 
strongj and would have resulted in complicated negotiations with 
Bradbury*s agent. It would also have involved a great deal of 
script*Ywriting, with a high cost in time and energy. Finally, 
the puFely theatrical aspects of the production were largely 
"extracurricular" and difficult to justify in terms of person- 
hours required to do the job. 


An alternative project we will propose can meet some of the same 


1. Provide a focused activity to stimulate and 
integrate development of the media room 
environment and media design and production. 
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Provide a demonstration of the technical 

and interactive capabilities of the media room 
that will stimulate creative collaboration with 
theatrical and performance artists. 


S. Create an occasion for group collaboration 
and celebration. 
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Our new. improved project will also meet some specific research 
goals more directly and will provide more generative results. 
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{In collaboration with Arthur Fischell and Alan Turing? 


The central goal of the F.T. Project is to see how a first- 
person, interactive fantasy system implemented in the media room 
might. work. We envision meeting that goal by creating (in the 
‘fine tradition of the Architecture Machine Group) a simulation of 
a single-user interaction with such a system. The project 
represents a convergence of the theories and techniques being 
developed in the media room project (Hulteen and others) and in 
the interactive story project (Marion, Laurel. et. als). The 
actual event will have the flavor of a planned improvisation, 
using operational technical elements where they exist and human 
improvisation for portions of the system that are still in the 
conceptual hacking stage. 


The project will utilize a structure which is based on the 
interactive story idea, with a playwriting expert system (PLAY- 
RIGHT) which enables first-person participation of the user 
CINTERACTOR) in the development of the story or plot, and which 
orchestrates system-controlled events and characters so as to 
move that collectively generated story. forward in a dramatically 
interesting way. The media room environment may be seen as the 
interface for the interactive story experience. 


The operational elements of the system will include some 
interactive devices, video, graphics, and animation, sound, and 
some other environmental effects. We will employ portions of the 
design (if not the actual code) of Meehan’s story generation 
program, Tale-Spin, as well as the design of the PLAY-RIGHT and 
other elements of the interactive fantasy version of that system 
(SEE attached system diagram and “Explanation of I.F. System" 
below). #° — 
Again, our purpose is to study the design of a system, major 
portions of which have yet to be implemented, by simulating its 
operation. The exercise will be improvisational in the sense 
that humans will enact roles that will, hopefully, later be 
automated -- much like "enacting" a machine which has some 
working parts, with humans filling in for the parts that are yet 
to be built. Such human participation is necessary simply 
because machines are not available that can perform these tasks, 
and yet we need to see how the system works before we can build 
those machines. One of our jobs will be to observe how well 
humans perform at a task which we hope to program computers to do 
later. We suspect that a person may not, for example, be able to 
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perform the PLAY-RIGHT task in real time. By identifying the 
nature of Both the difficulties and successes that our human 
PLAY-RIGHT encounters, we can use the information gathered to 
continue design of the I.F. system. 


The F.T. project will address the research goals mentioned in the 
first section of this paper, and it will also allow us to: 


1. Involve theatrical or perfomance artists 
(like Ray Bradbury or Chris Hardman) in the 
position of the PLAY-RIGHT, whose actions and 
heuristics may later be modeled in the creation 
of the expert system. 


2. Incorporate and make extensive use of many 
interactive devices and strategies. 


3S. Create visual, auditory, and other sensory . 
materials (relevant to our chosen subject 
domain) and design techniques for accessing 
and integrating those materials in real time. 


4. Refine the design of an interactive fantasy 
system, which may be implemented with a variety 
of interfaces, including the media room. 


Se Further sharpen our image of the media room 
by.exploring an interesting sample “application.” 











Proposed Structure of Interactive Fantasy System. 





a) 


Explanation of the Interactive Fantasy (1.F.) System 
(Refer to diagram, “Proposed Structure of Interactive Fantasy 
System. ") 


PLAY-RIGHT: Plans and orchestrates plot development. Material 
resources inciude the Potential, the Characters, and the Play- 
right’s model of the Interactor. Formal (heuristic) resources 
include the Poetics and rules pertaining to Style (or genre). 


STATE-GF-THE-WORLD MODEL: A passive element of the system which 
‘contains a computer model of the dramatic environment, including 
the history of the world as well as the current state of affairs, 
which is read from and written to by the Play-right, the 
Characters, and the Interface. 


CHARACTERS: Computational personalities that know their own 

nature and cam make choices and perform actions in the context of 

the State-of-the-World Model. They are also capable of accepting 

new orders (psychotherapy or enlightenment) from the Play~-right 

as the plot develops. In effect, their traits and pres 
predispositions are continually formulated by the Play-right. pee Lica, 


INTERFACE: Implemented in the media room for this project. The 
Interface is an active system which consults the State-of—-the- 
World Model and provides a dynamic, sensory representation of the 
action. Conversely, it reads the interactor’s outputs and 
transmits them.to the State-of-the-World Model for inclusion in 
the current environment. : 


POETICS: The dramatic rules and heuristics utilized by the Play- 
right in the construction of the plot. These include notions of 
dramatic form and structure, causal relations among the elements 
of the drama, methods for creating various types of incidents 
(suffering, discovery, and reversal), and means for creating and 
maintaining lines of dramatic probability. 
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POTENTIARS:.. Literally, all the materials of the story world, 

Parr) which aea@ecefined by a human author before the system is placed 

im motion. nly characters, objects, and events which are 

formulated from these materials can be included in the plot. 
Potential may be described to the system in various states of 
formulation3} for example, it may include general descriptions of 
the traits of beings which may inhabit the world and/or 
formulations of specific characters, depending upon the desires 
of the author. 


Vv U 
,~* STYLE: This "bax" may actually contain notions of form or genre 
hws f Ceeges tragedy, comedy, or melodrama) as well as functional 
ine PF. descriptions of styles €e.g., expressionism, realism, or neo- 
ped is classicism). It is unlikely that these functions will be 
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differentiated from the operations of the Play-right in the F.T. 
version of ‘the system. 


INTERACTOR: The (human) user of the system. He/she is allowed 
to interact with the system through the Interface, which controls 
access to all forms of human input and output. The Play-right 
generates a predictive model of the Interactor for the purposes 
of orchestrating the plot. Each Character also creates a unique 


model of the Interactor as part of his/her knowledge about the 
world. 


Successive Approximation: We envision several iterations of the 
F.T. experience, allowing us both to tweak the system design and 
to fill in obvious holes in areas like video and sound. Here is 
a rough outline of steps in the "pre-production" phase of the 
process: 


1. Meet with interested parties to discuss the 
overall project, then zero in on technical 

and creative needs. The theatrically-oriented 
be largely eliminated, but we hope that the 
technical and creative commitments made by 
various researchers to the previous project 
can be carried over into this one. . 


2. FPerson associated with each area of responsibility 
creates a cost estimate, which we will roll up 

and present to Kristina for analysis and approval. 
This will include equipment, person-hours, media, 
and consulting (primarily the PLAY-RIGHT person). 


3. Second group meeting with PLAY-RIGHT in attendance. 
Attempt to make a decision on the content of the 
“~~- story world. Based on both project goals and 
s resource requirements, create a schedule for the 

~ entire project. 


4. Design and development of story world, media, and 
environment. 


Here is a proposed version of the "running" phase of the project: 


1. Meet with PLAY-RIGHT person to discuss story and 
review media developed to date. 
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FIRST RUN: This will be the most improvisational 
session. It should allow us to create a version 





of the story that we will use in future runs. Tt 
will also serve to identify obvious flaws in the 
system design and hotles in the media which we can 


repair. We should begin videotaping with this 
session. 


ce SUCCESSIVE RUNS: We envision a finite number of 


successive runs, with the intention of creating . . 
aversion which is exemplary enough of our system 
that we may combine the videotape and some comments oe 

to create a viable concept presentation or "demo." me 
This demo will stand as documentation of the I.F. Sos 
project-in-progress and may also be used as an iS 


example of the media room in operation (it may 
also be used as an adjunct to Laurel*s thesis). 
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4. Future use! At the conclusion of the last scheduled _ . 
run, the project may be torn down or run again with <= 
different artists and/or story worlds. In any case, 
such future uses should be considered new projects. 


foot Sti 


eds ©, Characteristics of the project are some more subjective or 

et ade artistic Qualities which are equally valuable to our research. 3 

jade One of the objectives of the Veldt project was to employ the 7.8 

: media room environment in an activity that was centered around —. <: 
the arousal and purgation of emotions, as opposed to the rationals .. =: 
or objective representation of knowledge. The F.T. project, ae 
operating as it will within a fantasy domain and employing Sak 
dramatic techniques, preserves this focus. Such "dramatic" usage. < 
of the environment has as its outcome a kind of pleasurable ee 
experience that is associated with drama, fantasy, and 
imagination, as distinct from the pleasure that results from the 
satisfaction of curiosity or the acquisition of knowledge. 


Currently, it is productive to distinguish between these two 
types of 


eutcomes, especially on the levels of context a 
functicarimiaty. The end cause of an Raphael Wing vere aa 
experienc@ is presently conceived as distinc® from that of a 
session with an encyclopedia, and it necessary to explore 
the emotional and artistic dimensions of interactive media ina 
context where the existence or relevance of those dimensions is 
not a subject of debate. The interactive fantasy world provides 
that context. Of course, as we continue to work on both ends of 
the interactive spectrum, our understanding of the uses of 
emotion and aesthetics in the representation of knowledge will 
grow. As our thinking moves in that direction, it is not 
unreasonable to propose that the I.F. system be used as an 
interface to various knowledge domains. This is how a 


convergence of these two currently distinct lines of inquiry may 
occur. 
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The central goal of the proposed project is to see how a first-— 
person, interactive fantasy system implemented in the media room 
might work. We envision meeting that goal by creating a 
simulation of a single-user interaction with such a system. 


By "interactive fantasy" we mean a first-person encounter with a 
fantasy world, in which the user may create and portray a 
character whose choices and actions affect the course of events 
just as they might in real life. The structure of the 
Interactive Fantasy system, described in detail below, utilizes a 
Playwriting expert system (PLAYWRIGHT) which enables first-person 
participation of the user in the development of the story or 
plot, and which orchestrates system-controlled events and 
characters so as to move the action forward in a dramatically 
interesting way. 


The Media Room, the proposed environment for the project, is a 
prototypical human interface which is intended to include all 
available forms of person/machine communication. Current 
capabilities include speech recognition, speech synthesis, touch 
sensitive displays, rear screen projection, videodisc, 

: mputer graphics, body tracking, four channel audio, 
is, and a LISP machine computer. Planned additions 
wre eye tracking, additional body tracking, a real- 
time animation system, and digital audio capabilities. Olfactory 
and thermal effects may also be employed in the Interactive 
Fantasy simulation. 






The project represents a convergence of the theories and 
techniques being developed in the media room project (Hulteen and 
others) and in the interactive story project (Marion, Laurel, et. 
al.). The actual event will have the flavor of a planned 
improvisation, using operational technical elements where they 
exist and human improvisation for portions of the system that 
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have yet to be fully designed or implemented. 


Again, our’ purpose is to study the design of a system, major 
portions of which have yet to be implemented, by simulating its 


operation. The exercise will be improvisational in the sense 
that humans will enact roles, portions of which may later be 
automated -- much like "enacting" a machine which has some 


working parts, with humans filling in for the parts that are yet 
to be Built. Such human participation is necessary simply 
because machines are not available that can perform these tasks, 
and we need to see how the system might work before we can build 
those machines. 


One of our jobs will be to observe how well humans perform at a 
task which we hope to program computers to do later. We suspect 
that a person may not, for example, be able to perform the 
functions of the PLAYWRIGHT task in real time. Other functions 
in the system, such as the creation (authoring) of the fantasy 
world, may always be more elegantly and appropriately performed 
by humans than by machines. By identifying the nature of both 
the difficulties and successes that our human participants 
encounter, we can use the information gathered to continue design 
of both the human and machine elements of the Interactive Fantasy 
system. 


The Interactive Fantasy (IF) simulation project is designed to 
meet the following research goals: 


he Involve literary, theatrical, or performance artists 
in the position of PLAYWRIGHT, whose working methods 
and heuristics may be modeled in the creation of the 
PLAYWRIGHT and POETICS portions of the system. 


<.e Employ the talents of those same artists as authors 
or creators of the fantasy world itself, observing 
the problems encountered in order to design authoring 
techniques and aids that will facilitate such 







antributions. We also hope that the experience will 
rovide generative liaisons with such artists which 
Gan contribute to a variety of projects in the lab. 


Provide the opportunity for observation and analysis 
of the overall design of the IF system. 


Ms. Incorporate and make innovative use of various 
[ interactive devices and strategies. 
es : 


‘4. Create visual, auditory, and other sensory 
materials (relevant to our chosen story domain) 
and experiment with techniques for accessing 
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and integrating those materials in real time. 
These results should be generalizable to other 
projects (e.g., the encyclopedia project) which 
may Utilize the Media Room as an interface. 


Qs. Further sharpen our image of the Media Roam 
by exploring an interesting sample application. 
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(Refer to diagram, "Proposed Structure of Interactive Fantasy 
System. ") 


PLAYWRIGHT: Plans and orchestrates plot development. Material 
resources include the POTENTIAL, the CHARACTERS, and the 
PLAYWRIGHT’s model of the INTERACTOR. Formal (heuristic) 
resources include the POETICS and rules pertaining to STYLE (or 
genre). In the IF simulation, we expect that the same person who 
performs the PLAYWRIGHT function will also act as author of the 
fantasy world, a distinctly and permanently human task (see 
POTENTIAL, below). 


INTERACTOR: The human user of the system. He/she is allowed to 
interact with the system through the INTERFACE, which controls 
access to all forms of human input and output. The PLAYWRIGHT 
generates a predictive model of the INTERACTOR for the purposes 
of orchestrating the plot. Each CHARACTER also creates a unique 
model of the INTERACTOR as part of his/her knowledge about the 
world. 


STATE-OF-THE-WORLD MODEL: A passive element of the system which 
contains a computer model of the dramatic environment, including 
the history of the world as well as the current state of affairs, 
which is read from and written to by the PLAYWRIGHT, the 
CHARACTERS, and the INTERFACE. 


Computational personalities that know their own 






nature @ ‘an make choices and perform actions in the context of 
the 57 'THE-WORLD MODEL. They are also capable of accepting 
new orce 


Le psychotherapy or enlightenment) from the PLAYWRIGHT 
as the plot develops. In effect, their traits and 
predispositions are continually formulated by the PLAYWRIGHT. In 
the IF simulation, the CHARACTERS may be represented variously by 
human actors and machine-simulated personalities. 


INTERFACE: Implemented in the media room for this project. The 
INTERFACE is an active system which consults the STATE-OF-THE- 
WORLD MODEL and provides a dynamic, multi-media representation of 
the action. Conversely, it reads the INTERACTOR’s outputs and 
transmits them to the STATE-OF-THE-WORLD MODEL for inclusion in 





the current environment. 


POETICS: The dramatic rules and heuristics utilized by the 
PLAYWRIGHT in the construction of the plot. These include 
notions of dramatic form and structure, causal relations among 
the elements of the drama, methods for creating various types of 
incidents (suffering, discovery, and reversal), and means for 
creating and maintaining lines of dramatic probability. 


POTENTIAL: Literally, all the materials of the story world, 

in metion. Only characters, objects, and events which are 
formulated from those materials can be included in the plot. 
POTENTIAL may be described to the system in various states of 
formulation; for example, it may include either general 
descriptions of the traits of beings which may inhabit the world 
or formulations of specific characters (or both), depending upon 
the desires of the author. 


STYLE: This portion of the system might contain notions of form 
or genre (e.g., tragedy, comedy, or melodrama) as well as 
functional descriptions of styles (e.g., expressionism, realism, 
or neo~classicism). [It is unlikely that these functions will be 
differentiated from the operations of the PLAYWRIGHT and author 
in the IF simulation project. 


We envision several iterations of the IF simulation, which will 
allow us both to successively modify the system design and to 
fill in obvious holes in areas like video and sound. Here is a 
rough outline of steps in the pre-production phase of the 
process: 


1. Meet with interested parties to discuss the 
overall project, then zero in on technical 
and creative needs. 









~ Create and present for approval a cost estimate 
or all aspects of the project, including 
equipment, person-hours, media, and consulting. 





3. Discuss the project with selected artists and 
identify PLAYWRIGHT for first round of simulations. 


4. Meet with project team and PLAYWRIGHT. Select 
story domain. Based on both project goals and 
resource requirements, create a schedule for the 
entire project. 





un 


5. Design and develop story worid (POTENTIAL), media, 
and interactive environment. 


The running phase of the project will include the following 
steps: 


1. Meet with PLAYWRIGHT to discuss story and review 
media and environmental features developed to date. 
Refine and augment materials as necessary. 


2. Produce first run of the simulation. This will be 
the most improvisational session. It should allow us 
to create a version of the story that we will use as 
an example and for fine-tuning future runs. It 
will also serve to identify obvious flaws in the 
system design and holes in the media which we can 
repair. We should begin videotaping with this 
session. 


3. Produce successive runs of the simulation, with 

the intention of creating a version which is 
exemplary enough of our system that we may combine 
the videotape and some comments to create a viable 
concept presentation or "demo." The demo will stand 
as documentation of the IF project—-in-progress and 
may also be used as an example of the Media Room in 
operation. 


4. Analyze and evaluate results. At this point we may 
want to consider future runs with modified system 
design and/or different artists participating in the 
Simulation. 


In addition to its scientific value, the IF simulation is 
intended to explore some subjective and artistic issues which are 
equally mtelemvant to our research. One of the our objectives is 

: ' Media Room environment in an activity which 

wotional response. The IF system is intended to 

oo en@ user a kind of pleasurable experience that is 
associated with drama, fantasy, and imagination, as distinct from 
the pleasures that result from the achievement of functional 
goals or the acquisition of knowledge. 






Currently, it is productive to distinguish among these various 
outcomes, especially as they reflect the assumed relations 
between context and functionality. The end cause of an 
interactive fantasy experience is presently conceived as distinct 
from that of a session with an encyclopedia, and it seems 
necessary to explore the emotional and artistic dimensions of 





interactive media in a context where the existence or relevance 
of those dimensions is not a subject of debate. The interactive 
fantasy world provides that context. Of course, as we continue 
to work within the entire spectrum of interactive forms, our 
understanding of the uses of emotion and aesthetics in the 
representation of knowledge will grow. It is not unreasonable to 
propose that the IF system may someday be used as an interface to 
various knowledge domains. This is how a convergence of these 
two currently distinct lines of inquiry may occur. 
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Orientation 

What follows is a pastiche of conversations and descriptions 
drawn from an imaginary run of the IF Simulation (SEE "Research 
Proposal: Simulation of an Interactive Fantasy System"). 
Through the conceit of dialogue, this paper is intended to give 
the reader a sense of what the simulation might really be like, 
and to address some of the questions and issues that are sure to 
arise during the course of the project. IT have used the world 
created in Ray Bradbury’s book, SOMETHING WICKED THIS WAY COMES, 
as the context for the imaginary events described here. Readers 
whe are not familiar with the story may obtain a synopsis from 
me, or may borrow the book. 


Pre-Production 


ATARI: What does the town look like where the action takes 
place? 


BRADBURY: Well, there are five versions of the story itself that 
will give you a sense of the locations and landscape-- the 
original version of THE DARK CARNIVAL, the screenplay based on 
that version, my novel, SOMETHING WICKED THIS WAY COMES, the 
"Play I wrote for Disney, and of course the film 
-effects people at Disney did a great job of Building 
‘designing the landscapes used in long shots. The 
actual is a collage of a hundred places, all with a 
certain ling in common, sprinkled across northern Illinois, 
some places in Ohio... 








A: Sounds like plenty of resources. If we can obtain some of 
the Disney footage it would be great, but it also sounds like we 
know where to shoot if we decide to do it ourselves. What about 
the carnival? 


B: There’s always a danger in making things too concrete. 
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Again, Disney did a great job of obscuring things, bending the 
light, focusing on the mysterious-~ they did a wonderful job with 


the train. for instance. It*s a typical midwestern traveling 
show, but "through a glass darkly"-- the tents are black, the 
facades and posters are ominous-—- and of course you can smell it 


and feel the pricking of your thumbs—-— 
A: That’s interesting-- how would you describe that smell? 


* © & & 


Az: ...s0 the action would probably begin in that upstairs 
bedroom, with the sound of the approaching train, and the light 
blinking on in Tom Nightshade’s window-- does that sound right? 


BR: Yes, I think we could begin there. So You <the Interactor> 
might open the window and talk to Tom, or get frightened and 
snuggle down into the shadows-— 


A: Or call Dad-- 


BR: Or climb down the trellis and run off through the wisteria, 
toward the railroad tracks... 


x ee 
RB: How can we be sure You will make it to the Carnival at all? 


A: I?m not sure we need to. Maybe You try it six or seven times 
and then You just can’t stand it any more-—- You have to go and 
see-- 


B: Yes, and we can probe your imagination with the incidents we 
create in the story. Is the sound of the train not enough? The 
smell? A certain whisper in the air? Or perhaps a leaflet blown 
in the window-—— 


adqht. As the Playwright, you can do that-- keep 
dents that stimulate curiosity until You can’t stand 





A: Yes, we can use eye tracking to cue magical transformations. 
When You look away from that block of ice, for instance, the 
movement of your eyes can immediately trigger the new image, so 
that when You look again, the Dust Witch is there, inside the 
ice... 


HB: Wonderful. Couldn’t we use distorted video to create the 
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hall of mirrors in real time? 
Az: Sure-~ and work the transformations with computer graphics... 


* * & He 


BE: We hope that something different will happen for each person 

who enters the fantasy world-- and each person’s experience will 

be a different length, too. Suppose somebody wants tao just stay 

in there until we run them out. How does the user know when he’s 
done’? 


A: Well, we eventually want our Playwright program to know 
enough about the nature of endings in drama to be able to invent 
one. This means that the program (you, in this case!) has to be 
able to identify the central action of the story -~ the spine of 
the plot. Of course that spine may be different for every trip 
through the fantasy world. 


BR: Yes, I can imagine that. Say in one version that our user 
gets to the carnival and, through a series of incidents, becomes 
convinced that he must reveal Dark’s identity to the whole town. 
The story must move so that he is finally able to do it. 


A: Or not, definitely not. 
BRB: Yes, that would be an ending, too... but it wouldn’*t be as 
much fun. But how will your program know that that’s what that 


particular user if focusing on? 


A: Well, how will YOU know? 


B: (Chortling.) I see what you’re getting at. Well, I*1li watch 
what he does and listen to what he says and try to infer his 
intentions... 





Media Room) The bedroom looks great. I 
especially like the moonlight splashed across the floor over 
here, and the cloud shadows-- 


AS (Naimark) It’s amazing what you can do with scrim-— 
B: Now I’m walking over to the window-- I want to open it and 
look out-- what do I do”? 


A: (Hul teen) Go for it-- 
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(HRADBURY approaches the window and reaches toward it. Sound of 
window opening. The wall he is facing dissolves to the view 
outside the window. He turns back to see that the room is still 
represented on the remaining walls. He backs away from the 
window a few steps-- we hear it close, and the wall restores an 
interior view.) 


BH: Okay. I*m going down the trellis to the yard. 
(He "opens" the window again.) 

BR: What do I do now? 

A: (Laurel) Jump! 


(Hradbury takes another step. The screen becomes a blur. Sound 
of feet hitting the grass. Bradbury turns. Behind him is the 
wall of the house and the trellis: before him is the yard. Night 
sounds grow louder, and there is a scent of wisteria. Bradbury 
stands sniffing. There is the sound of an approaching train in 
the distance, coming from the left.) 


BR: Here it comes. I want to run, but T1711 hit the wall. 


A: This is a tricky one. Sweep your arm in the direction you 
want to go, and try to indicate the speed you want to travel with 
the movement. Like throwing a ball, or commanding a battallion 
to mave "Onward." 


BR: (Gestures rapidly and forcibly with his left hand. The yard 
whirs by, then the woods, until there is simply a blur). Yiiii! 
How do I stop? 


A: (Marion) Gesture! 






_animation sequence of the Dust Witch searching for 
No, she is much more... diffuse... much less explicit 
What you’re showing me looks like a Halloween cliche. 
: What does she look like, then? 


B: She's a color-—- Disney got it pretty well, I think-- a sort 


of ghastly green aura that moves around-- leaving silver trails 
where her fingers touch the roof, the trees-- seeking-- And when 
we do glimpse her, she’s incredibly old-- receding from life as a 


mummy does, receding from the senses~— 
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A: So what we need is more like ambient light and visual 
textures to communicate her presence-- and we can use computer 
graphics to paint the trails. 


BH: Yes, I think so-- and the sound of the fingers scraping, and 
of the air in the balloon-- breathing, almost-- but dry, dead 
sounds... 


* & & & 


{The whole crew stands inside the "control room" built for the 
Playwright.) 


A: These displays indicate what’s going on with eye tracking, 
body tracking, pointing and gesturing. The body tracking display 
is the best-developed: if we find out that any of the others is 
inadequate, we will default to video feedback. You"ll be able to 
hear what the speech recognizer understood with these earphones. 


B: What if the recognizer doesn’*t understand something? 


AB: It often won’t~- but then, people often don*t, either, in 
regular conversation. Let*s try it. If we can*t get enough 
information to make good guesses from context, we can default to 
live audio for the simulation. The idea, though, is to work 
within our technical constraints whenever we can. That*s why we 
hope you'll be able to use these displays instead of an actual 
picture of the user moving about the room. 


* & & & 


B: Okay. My job, as I understand it, is ta weave the story 
around the person walking around in the world we’ve created. 
What are my tools? 


A: Well of course your most valuable tools are your imagination 
and your skill as a storyteller. You know what a goad plot is 
like, am@cyou know how to motivate characters by throwing 
inciden id contexts at them. The "user" af our fantasy could 
be thoug as just one of those characters, wandering around 
in your dapyination-- self-motivated to the extent that he is 
alive for you-~- 







Br -- yes, and characters often surprise me as I*m writing 
stories-~- 
A: -—-- and yet you manipulate them through the settings and plots 


you weave; you poke at them to see what will develop. 


é 


B: Yes. And of course sometimes there are specific incidents 
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inevitable... 


A: That’s what we*ve tried to capture in some of the fixed 
sequences we’ve created-- jumping out of the window, running 
toward the train, moving through the hall of mirrors, or hiding 
from Mr. Dark in the Library. 


B: Those are all pretty big chunks of action, though. Aren*t we 
losing flexibility? 


A: Yes and ne. You are losing some flexibility in the internal 
structure of those incidents. But you gain some ease and scope 
in changing plot direction. In that sense, these big chunks 
increase overall flexibility in the plot. And you have some 
little pieces, too... 


*% © &* EF * 


Bs: But we can*t simply try to make our "user" just like Will-- 
that wouldn’t he any fun. 


A: No, but we know what kinds of transactions with this world 
would be really interesting from a dramatic point of view. We 
can try to create incidents that will make it probable for the 
user to make certain kinds of choices-- to orchestrate the action 
so that those essential Will-like qualities are drawn out-— 


B: Yes-~ curiosity, daring, an attraction to the Dark that is 
balanced by areal joy in living-- 


A: And so, as Will wandered around that little town in your 
imagination, as you were writing the story, what kinds of things 
did you do to give him the opportunity to express those 
qualities? 


=: The conversations with Tom Nightshade are one example—-- the 
‘hat Tom previded in every choice that the boys made 





Toeer" Gs more like Tom than Will? 





B: Then the complement he needs is more like Will than Tom... 
eee He * 


A: We want to get at the "rules" you use toa guide a plot in an 
interesting direction. 


Bs (Drawing himself up.) Well, I certainly wouldn’t consider 
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myself a formula writer-- I’m not sure f use any "rules"-— 


A: That"s precisely why we want to observe how you do this. 

When we design expert systems. we try to include not only the 
explicit knowledge of an expert but the "rules" by which he makes 
good guesses-- his hunches and intuitions, if you will. 

Sometimes they form patterns, and by observing them we-- and you- 
~ may discover some of the heuristic "rules" you use in doing 
your work. 


BS ~.ehmmmm... 


AS »~2eand sO we want you to do your best to talk us through each 
decision you make, at least the first time through, so we can get 
a Glimpse of those processes. 

U 


Bs Well, I*1ll try. 


A: We’*re also trying to get at categories of things that might 
live in your "toolbox" -- kinds of incidents, for instance. When 
is it time in a story for a surprise? 


Be I see what you mean. Well, the boys get a Surprise when Mr. 
Dark shows them his "living tattoo.” It functions as immediate, 
sensory evidence that Dark is extraordinary and powerful and 
takes the action to a higher level of suspense. 


A: And a "reversal" might be another category... 
BR: Yes, an incident that completely changes your expectations... 


* &* & & * 


Bs I think I understand what you want me to do in constructing 
the story. But how does it become real for the “user?” 


“room” is going to respond in a number of ways, as you 
are a limited number of slides and video sequences, 
he computer graphics and lighting effects we looked 
You also have the sound library and some capability 
w sounds as you need them. In addition, you have the 
"electronic personalities"-- the other characters-- who can be 
made to speak and act, within some technical constraints. 







B: Yes, you showed me the Dust Witch and Mr. Dark 
“nersonalities." I can control general placement and gross 
movement with the joystick, and can have them speak dialogue. 
But it’s entirely likely that other characters will need to make 
an appearance-- maybe even some who weren*t in the original 
story. 
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A: We know that, and I’m afraid we aren*t far enough along to be 
able to create them for you on the fly. We thought we’d try to 
have it both ways for the simulation, so that you could at least 
have the experience of creating a new character if you need one, 
and so that we could observe how you do it. In order to do that, 
we've plugged another human into our simulated machine. He's a 
professional actor, and you can talk to him through the 
microphone on your headset. 


B: You know, this is beginning to remind me of Niven and Barnes* 
DREAM FARE... 


A: Yes, DREAM PARK was a real inspiration for us, just as your 
own story THE VELDT was... These are pretty powerful ideas, and 
it shouldn’t surprise us that a lot of us have had them... 


BR: (Peers over the top of his glasses.) 


First Run of the Simulation 


— ree vies cit —_——_—— —<— —— oe Re Oe ee SD Se oe ee ee 


NOTE: Statements in parentheses () are descriptions of events. 
Statements in brackets <3? represent unspoken thoughts. 
Statements in CAPS describe actions that occur between the 
"fleshed out" segments of the interaction. The USER is dencted 
as U, the INPUT to the Playwright in the control room as [, the 
PLAYWRIGHT as FP, and the OUTPUT af the media room as 0. These 
various views of the action are also successively indented, so 


that a single view may be followed sequentially. 


* &* & # 


U: <It leoks like I’m in a bedroom-- 1°11 assume it’s mine. IT 
wonder what"*s out the window. > 


: 
" wan 


(Bacly tracking display indicates user moving toward 
projected in north screen.) 






es 


She’s moving toward the window. Get ready to open 
Se Wait for the gesture. 


Us (Reaches toward window. } 


I: (Polhemus display indicates gesture toward window: body 
tracking display shows another step north.) 


P> Fire the window opening sequence. 


OQ: (Window opens in video image on north wall. 
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Sound of window opening. Night sounds become 
louder. Wisteria smell is dispersed.) 
Us <Wow' A summer night... looks like a small town... 


* © & He F 


THE USER MOVES INTO THE YARD, HEARS THE PASSING TRAIN, AND 
DECIDES TO WANDER OFF TOWARD THE TRACKS. 


%* *& % * % 
QO: (Circus train rushing by on east screen. 
Engine roar doppler-shifts as the train seems to 
mave southward. Train passes out of view; empty 
tracks illuminated in moonlight. >? 


Us A carnival ! 


Is (Speech synthesizer recieves ocutput from recognizer and 
produces phrase, "A carnivore!") 


Ps: Must be "carnival"-- She’s got it, but she’s not 
moving. Let*s try a strange glow in the southern sky. 


O: (Red and yellow searchlight beams produced in 
computer graphics sweep the sky on the south 
screen.) 


Us <<What*s going on over there?> 


P:; She’s still not going for it. Give me some 
carnival music. 


Oo: (Distant calliope music drifts in from the 


south. ) 
Us <Oh yo God, they’re setting it up-- in the middle of the 
night- yw got to see this.> 






Body tracking registers a few steps souths polhemus 
shows a moderately accelerating, sweeping gesture toward the 
south with user’s left hand.? 


P: Okay. she’s running at a moderate pace. Let’s go 
to the carnival. 


* & & & 


THE USER RUNS TO THE CARNIVAL SITE AND SEES THAT THE TENTS ARE 
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BEING SET UP. SHE NOTICES THE FORM OF A SMALL BOY IN THE 
SHADOWS. . 


* 4 & & 


Us Who are you? 


0: (Electronic character speaks: "It's me, Tom, 
silly. Your best friend, remember?") 
Ur: <¢I"°11 play along with this, I guess. I want to get closer to 
the action.* Oh, hi. Hey, let’s sneak over there and have a 


look around, okay? 


i: (Speech synthesizer blurts, "Hi... sneak over... look 
around, ofay?") 


PF: Good. We’ve got a lot to process here-- let’s have 
Tom take the lead and get them there slowly, from the 
west. 7 

O: (Tom: “Sure, follow me." Tom moves slowly. 


across the south wall as carnival tents grow 
closer on the east screen. Calliope music grows 
louder.) 


* * &F Ee F 


TOM AND THE USER HAVE MOVED UP TO A LARGE TENT REPRESENTED ON THE 
EAST SCREEN. THE USER SUGGESTS THAT THEY LOOK INSIDE. THEY 
DISCOVER THE CAROUSEL. JUST AS THEY APPROACH IT, MR. DARK 
APPEARS FROM BEHIND THEM, INTRODUCES HIMSELF, AND ANNOUNCES THAT 
THE CAROUSEL IS BROKEN. WHEN DARK HAS GONE, OF COURSE, THE USER 
DECIDES TO CLIMB ONTO THE CAROUSEL. 


* * & % F 


Q: (The carousel is shown on the east screen, 
while the other screens reveal interior surfaces 
of the tent. Tom is displayed on the north 
screen, near the carousel. The night sounds have 
resumed following Mr. Dark’s exit.) 





U: I wish we could ride it. <I wonder if there’s any way to 
climb onto it.> 


bE (Body tracking shows user approaching very near east 
screen. Speech synthsizer mumbles, "wish... ride....”) 


Ps Let*s get her on the carousel-- 
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0: (Sound of feet hitting metal as user steps 
toward east screen. Images on other screens drop 
down an apparent two feet. Tom: "Hey, Mr. Dark 
Said you’re not supposed to be up there!") 


Us: (Walks southward "around" carousel.) These horses sure look 
scared. (Turns back to Tom.) 


se (Body tracking registers user*s movement. Speech 
synthsizer says, "horses... look scared.'") 


P: Good, she’s moving around it and she’s beginning to 
get an ominous feeling. Adjust the images so she can’t 
see Tom. Make it generally darker to increase the 
Suspense. 


0: (West wall is completely darks other walls 
filled with views of the carousel from its edge. 
Footstep sounds continue in sync with user’s 
steps. Night sounds stop abruptly.) 


Us Tom”? 


eH (Synthesizer says, "“Tom?" Galvanic skin response 
display shows a sharp jump. Body tracking shows that user 
has turned to face out toward the darkness and has taken a 
step "away" from the carousel.) 


P> She’s getting scared, and she’s trying to get off. 
I didn*t expect that. Start the carousel with a jolt. 


Os (Loud mechanical sound of low frequency shakes 
the floor. Loud carousel music, with speed and 
vGlume gradually increasing. East wall continues 
to display carousel while streaks of color blur 
across the blackness on the other screens ta 
indicate accelerating rotation. Image on east 
screen is gradually replaced with a blinding 
yellowish glow. ) 





Us TOM! MAKE IT STOF!!'! 


* *£ & He 


TOM STOPS THE CAROUSEL. THE USER STUMBLES BACK TOWARD THE CENTER 
OF THE ROOM AND NOTICES THAT SHE IS MOVING SLOWLY IN RELATION TO 
THE IMAGES ON THE WALLS. HER VISION SEEMS BLURRED AND SOUNDS ARE 
LESS DISTINCT. 
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0: (Tom: "Oh my God, Susan. You're... OLD!") 
Ur What do you mean? Why can’t I hear you very well? 


P: Let*’s give her some evidence. Use eye tracking 
output to place a "mirror" within her view. 


O% (Tom follows the user to the "mirror." Tam: 
‘took! " Video image in the mirror frame is the 
face of an extremely old woman.) 


*% © & & 


TOM AND THE USER ARE DISCOVERED BY MR. DARK. AFTER A CHASE 
THROUGH THE SHADOWS AND TRAILERS OF THE CARNIVAL THEY MANAGE TO 
RETURN TO THE CAROUSEL. TOM HAS SUGGESTED THAT RUNNING THE 
CAROUSEL BACKWARDS MIGHT REVERSE THE EFFECT. THE USER JUMPS ON 
AND TOM OPERATES THE CONTROLS. 


*% & & KF 


QO: (The carousel image is once again stationery 
on the east screen, while colors course across the 
others to indicate counterclockwise rotation at a 
moderate rate. The music plays backwards.) 


Ps This is too easy. Let’s have one last surprise. 
(Moves Mr. Dark into the scene next to Tom.) 


QO: (Dark: "I told you the carousel was BROKEN!" 
Dark grabs Tom and takes the controls. The 
carousel motion increases drastically.) 


Ue TOR (Begins to move around the room.) 


I: (Body tracking shows user moving around. GSR is going 
nut. 





opt What’s she doing? I can’t make sense out of the 

body tracking display. Let*s ga ta video-- okay, it 
looks like she’s trying to find a way out. Let’s turn 
up the gain. 


Q: (Tom: NHE?S GOT ME!!! GET OFF THE 
CAROUSEL!!!" Carousel continues to accelerate. >? 


WJ: (Runs around the room trying to find a way off the carousel.) 
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1; (Body tracking shows user moving rapidly around the 
room. Polhemus shows frantic gestures.) 


P: Ignore the gestures. Surround her with the 
carousel for five seconds then use the body tracking to 
keep an open screen just out of reach. 


0: {All screens show carousel with superimposed 
whirling colors. After five seconds, a blank 
screen begins to “anticipate” the user’s 


movements. Tom’s voice comes from the blank 
screen: "Jump! Susan, do you hear me? JUMP!") 
WU: <If I stay on here much longer I don*t know what will happen. 


The only place to jump is into that void that keeps showing up 
over there, where Tom’s voice is coming from.> 


I: (Body tracking shows slower movement; frantic gestures 
are still indicated.) . 


Pi She’s getting tired. Let her catch the window. 


QO; {Motion on all screens slows gradually. Tom: 
"Jump! PLEASE jump! ") 


Us (Runs toward the blank screen.) 
Ii (Rody tracking shows rapid approach to west screen.) 


ee Now. 


0: (Carousel is shown madly whirlirig on east 
screen. Other screens are now stationary images 
of the interior of the tent. Tom is near the 
controls on the south screen. Dark is running 
toward the user, superimposed over the east 
screen. Tom: “PUSH HIM! SUSAN! PUSH HIM! !!'") 


or a moment then runs directly at the east screen. 
“@f Dark looms in the center, directly over a touch- 
P@isplay. User pushes on the screen.) 





I: (East TSD registers user contact.) 


Ps All right, she*s pushed him onta the carousel. 
That should do it. Now let’s get them out of there. 


O: (Tom: "Leave him! That will finish him! 
Let*s get out of here!") 
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U: (Gestures toward the opening in the tent on the north 
screen.) That way! 


P: Now it’s time for a final twist. You don*t mind, 
do you? 


0: (The screens show the countryside landscape at 
a blurring pace, with the carnival receding in the 
west. Suddenly the motion stops and the user is 
once again in the interior of the bedroom, as 
before. ) 


* & & & 


— a a coe ce ts 


It may concern the reader that this particular experience would 
probably scare the hell out of the user. The potential for the 
arousal of fear is high in this particular world (and in many of 
the worlds that might be created by Mr. Bradbury). We should 
imagine that a user will ultimately be able to choose from among 
several different worlds, some of which are more benign. By 
choosing this one, our user has expressed a taste for horror, and 
the system has responded appropriately. 
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Abstract 

This is the first in a series of papers employing 
dramatic theory in the design of a computer-based system 
that ic intended to create interactive fantasy experiences. 
While the deSign process will reflect the methodology of 
computer software design and must be informed by the 
techniques and limitations of the technology, the central 
activity will be the application of the principles of 
Aristotelean dramatic theory to interactive structure. This 
paper will provide a description of bie proposed 
"Interactive Fantasy System™ and an overview of the 


functional requirements of such a system. 











CONFIDENTIAL 2 3 November 1983 


1. Overview of a New Form 


Dad The Idea of Interactive Drama 

A young woman sits before a personal computer console 
using the joystick to maneuver her starship into firing 
position as she fends off a simulated attack by a swarm of 
Zylon vessels. When the attackers are destroyed, she taps 
the keyboard to display a map of her quadrant of the galaxy 
on the screen to determine where other concentrations of 
Zylons are awaiting her. For now, she is assuming the role 
of captain of a starship, with the goal of defending the 


1 
galaxy from the an alien onslaught. 


In Niven and Barnes' science fiction novel Dream Park, 
adventure game aficionados pay hundreds of collars apiece to 
participate for two days in a simulated adventure. Thev 
assume characters with various magical powers and 
instruments, drawn from the lore of adventure gaming. They 
encounter holographic monsters, explosions, and hoards of 
murderous ghouls as they seek treasures in the world of the 


2 
Cargo Cults of the 1930's. 





1 
"Star Raiders," home computer game (Atari, 1979). 


2 
Larry Niven and Steven Barnes, Dream Park (New York: Ace 
Books, 1981). 
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There exists in our culture a number of deeply felt, 
shared fantasies of the kinds of experiences we might have 
with high technology. The idea of first-person, interactive 
excursions through imaginary worlds is one such powerful 
fantasy, which has been expressed in literature, film, and 
popular culture for decades. It is at the root of the 
desire which is evoked but only partially fulfillea by the 
video game fad. The wished~for experience might be compared 
to volitional dreaming, or to the idea of becoming a 
character in a play, affecting the action and outcome by 
making choices and performing actions in the imaginary world 
of the drama. What would it be like to be Hamlet or Captain 
James T. Kirk? “How would it feel? How might one co things 
differently than the characters who have already been 


created? What could one learn by doing it? 


Such an speeeienes would afford the uSer pleasures which 
are both similar to and distinct from those offered by 
viewing or writing a play or by playing a video game. Like 
an audience member, the user could experience the “willing 


of disbelief," allowing himself to participate 





y in an experience without having to cope with any 
real-world consequences. Like a writer, he could control 
the thoughts and actions of a character and create or 


influence various events in the environment, and could have 


the pleasure of creating a new work of art. Like the player 
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of a video game or role-playing game, he could imagine 
himself to be actually involved in the experience as an 
agent by assuming a fictitious role. A playwright may in 
fact have similar experiences "in his head” as he works on a 
sCript: he may "enact" his characters in his mind, taking 
vicarious pleasure in choices, actions, and their 
consequences in an imaginary world. No form of experience 
currently exists, however, in which one could hope to 
partake of all these pleasures at once in any world more 


palpable than that of the imagination. 


1.2 Critical Qualities of Interactive Drama 

Using the theory of the drama and the tools afforded by 
computer technology, it is possible to imagine a_e system 
which would fulfill the fantasy of first-person, dramatic 
interaction in an imaginary world. What are the critical 


qualities of such a system? 


First, the experience enabled by the system must be 


interactive. Interaction may be defined as "mutual or 
3 
reciprogal action or influence." A fully interactive 







eis the product of ongoing collaboration between 


cipal participants. The principal participants in 


3 
= Dictionary (New York: Pocket Books, 


The j 
1974), p. 372. 
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the proposed interactive experience are the human user and a 
System animated by artificial intelligence. The interactive 
requirement means that the user and the system must have 


mutual or reciprocal influence on the action that occurs. 


Second, the experience must be dramatic; that is, it 
must be productive of the pleasure associated with drama. 
That pleasure arises from the selection and arrangement of 
incidents and from the organic beauty of a whole. Drama 
does not provide an imitation of life in all its redundancy 
and confusion, but rather presents an imitation of action 
which, through artistic formulation, excludes unnecessary 
detail and complication and makes visible the causal 
connections among events. The dramatic form functions to 
maximize the pleasurable ons, of emotion on the part 


of the audience (or "user"™). The system, then, must be 


capable of structuring the experience in dramatic form. 


Finally, the user must be able to interact with the 


system within the context of the experience itself, assuming 





9g) a role within the fantasy world. This quality 





scribed as “first-person experience,” and it 


4 


Aristotle, Poetics, trans. Ingram Bywater, in Rhetoric 
and Poetics of Aristotle (New York: The Modern Library, 
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requires that the user's suspension of disbelief never be 
interrupted by attending to the system Which is "behind" the 
fantasy world. Informational questions, error messages, and 
explicit prompts are examples of such interruptions, and can 


be characterized as "second-person" transactions. 


The first-person requirement is best met by casting the 
user in the role of an agent, or character, within the 
fantasy world. This solution is also ideal in meeting the 
dramatic requirement of the experience: the user-character 
may be treated by the system, with some additional levels of 
inference and predictive modeling, as just another agent in 
the action being imitated. Thus the user's actions may be 


more eaSily integrated into the dramatic form. 


1.3 The Interactive Fantasy System (IF) 

"Interactive drama," then, is a first-person experience 
within a fantasy world, in which the user may create, enact, 
and observe a character whose choices and actions affect the 
course of events just as they might in a play. The 


structure of the Interactive Fantasy System proposed in this 







pape Alizes a playwriting expert system that enables 


firs son participation of the user in the development of 


the story or plot, and orchestrates system-controlled events 


and characters so as to move the action forward in a 
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5 
dramatically interesting way. 


The’ name, "Interactive Fantasy System," was chosen 
because it better suggests the critical qualities of the 
experience than the term, “interactive drama." To the 
uninitiated, “interactive drama” might refer to an 
interactive play-authoring system, and does not necessarily 
suggest the first-person quality of the desired interaction. 
Fantasies are typically first-person affairs, with 
emotionally satisfying structures and outcomes that are 
reminiscent of the drama. (The acronym "“IF" is also more 
appropriate to the system than the alternative, "ID.") The 
functional requirements of an IF system are discussed in 


section 2. 


1.4 Antecedents 

Several forms which combine notions of interactivity and 
Gramatic experience have contributed to the conception of 
the IF system. The three principal antecedents of the 
system are "traditional" drama, certain artificial 


programs, and certain interactive computer 





a 


Brenda K. Laurel and Eric A. Hulteen, "Simulation of an 
Interactive Fantasy System," research Proposal, Atari 
Corporate Research, 1983. 
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1.4.1 Interaction in Traditional Drama 

A play may be defined as an imitation of an action which 
is represented in a dramatic manner; that is, it is enacted 
by pee SnTece in real time, "as though they were actually 
doing the things seen During the performance of a 
play, the relationship between the audience and the 
performance is severely constrained. The audience views the 
performance and may experience and investigate emotional, 
kinesthetic, and rational responses to the ongoing action. 
An audience member may communicate the gross nature of some 
of those experiences to the performers in the form of 
audible responses (laughter, applause, etc.). Those ‘audible 
responses may affect the performance of the actors on stage, 
but will not, except in experimental works which are 
intended to be participatory, affect the course of the 
dramatic action or change a single line of dialogue. Beyond 
the secondary effects of audience responses upon the actors 


in the performance, then, no interaction between the 


audience and the play occurs. 


The absence of interaction between the audience and the 





play 8 necessary in order to preserve the dramatic form. A 
ape 


play is an imitation of a whole action, accomplished through 


6 
Aristotle, Poetics, p. 226. 
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the representation of a number of causally related 
incidents, each of which stands in organic relation to the 
whole. The selection and arrangement of those incidents is 
essential to the drama: 

The truth is that, just as in the other imitative 
artS one imitation iS alwavs of one thing, so in 
poetry the story, as an imitation of an action, must 
represent one action, a complete whole, with its 
several incidents so closely connected that the 
transposal or withdrawal of any one of them will 

7 
Gisjoin and dislocate the whole. 
Interaction between the audience and the play must therefore 


be prevented in order to preserve the integrity of the play 


itself. 


While interaction between the audience and the play at 
the time of performance is proscribed, there are occasions 
in which interactive play-writing can be successful. Actors 
in improvisational forms like the Commedia dell'Arte are 
required to have sufficient knowledge of Gramatic form and 


Structure so as to create viable plays in real time, thus 


the actors' relationship to the play may be described as 


wrTA toe 





tive. One might imagine an audience member or critic 


o's 


suggestions for changes in a play to the playwright 


during the revision process. The critic is thus interacting 


: 
Ibid., p. 233. 
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with the play via the playwright. The thing that such 
examples have in common is the participation of a 
playwright, or the introjection of playwriting expertise, 


into the interactive process. 


1.4.2 Child's Play and Role-Playing Games 

Children at play exhibit a facility Similar to 
playwriting expertise when they collectively create and 
maintain an internally consistent fantasy world for a whole 
afternoon (or series of afternoons). As Aristotle observed, 
much of children's play is imitation. The expertise which 
both maintains the integrity of the fantasy world and 
enables each participant to create, seemingly effortlessly, 
characters that are "good, appropriate, like, and 
consistent,” is in fact humanity's native skill in 
pneseien Although lacking the formal structure of drama 
ana thus its distinct emotional and aesthetic qualities, the 


imitative Play of children will serve as a powerful 


predictor of the behavior of human users of an IF system. 


A highly structured form of imitative play, known as 





@le-playing, is structurally quite Similar to an IF 
Dungeons and Dragons,” the first role-playing game 


to enjoy wide popular appeal, seems from its inception to 


g 
Ibid., pp. 226-7. 
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have been especially popular with computer professionals and 
enthusiasts, and computer-based versions began to appear 
almost immediately {the genre of computer-based “adventure” 


games was the result). 


Traditional (non-computer-based) fantasy role-playing is 


a collaborative affair: 


A fantasy role-playing (FRP) qame is one wherein 
the players construct characters who live out’ their 


lives in a specially made game-world. The 
characters need not be anything like the people who 
play them. . . . The game world is operated by a 


referee (sometimes called a game master, adventure 
master, dungeon master, etc.) who sets up the 
Situations which the flayers confront and who also 


plays ‘the world.' An FRP game, then, is an 

interaction between players, who operate (run) 

characters, and a referee, who runs the world in 
9 


which the adventures occur. 


The bulk of the “rules” by which an FRP game is played 
consist in the fantasy world itself, and are either drawn 
from some documented lore (as in most FRP kits and books” on 
the market today), arrived at consensually by the players, 


or a combination of both. 





‘games differ from an IF system in terms of 


9 
Greg Stafford and Lynn Willis, Basic Role-Playing: An 
Introductory Guide (Albany, CA: Chaosium Inc., 1980), p. 2. 
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enactment, personness, and dramatic structure: 


Most of the play is verbal exchange. The players 
tell the referee what they wish or intend to do. The 
referee then tells them if they can or may do it 

10 
and, if not, what happens instead. 


The experience coes not involve the elements of 
spectacle and music as a dramatic enactment would, nor does 
it occur in the first person, but rather through the medium 
of second-person transactions with the “referee. " The 
beginning and end of the game for each player are unique, 
defined by the creation and death of the player's character, 


and not by the shape of any whole action. 


1.4.3 Story Generation through Artificial Intelligence 

In the domain of narrative forms, computer-based 
interactive authoring environments have been created. 
TALE-SPIN, created by James Meehan at Yale University, is 
Gescribed by its author as a program which "makes up stories 
by simulating a world, giving characters some goals, and 
telling us what happens.” TALE-SPIN employs techniques of 
. intelligence to generate stories, and allows a 


to interact with the authoring process by 





"much of the information about the initial state 


10 
Ibid. 
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of the world, such as the choice of characters and 
relationships between one character and another." [In one of 
its modes of operation, the program Simply reports 
everything that happened in the "story" which was generated 
on the basis of the given characters and their  scoals. In 
another mode, the program employs narrative expertise to 
eliminate obvious and redundant information and descriptions 
of events which had no interesting consequences. In its 
most interactive mode, the program can produce "Aesop-like 
fables" to illustrate a "“moral™ (e.q., "Never trust 


ll 
flatterers”") which has been selected by the user. 


One might imagine a similar program which could 
construct plays rather than stories, thus providing an 
interactive playwriting environment. Just as TALE-SPIN 
produces stories which may be reac later as any other. story 
might be read, a play-writing program might be made to 
produce simple plays which might later be read or enacted. 
The play would be the result of an interactive authoring 
process, but the play as experienced by an audience would be 


nteractive than any other “traditional” play. 





ll 
James Richard Meehan, "The Metanovel: Writing Stories by 
Computer” (Ph.D. dissertation, Yale University, 1976), pp. 

15-6, 
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1.4.4 Interactive Computer Programs and Video Games 

The. Similarities between certain types of interactive 
sonuuver programs and drama are striking, and they provide 
avasipies Of dramatic elements in interactive contexts. 
There exists a class of interactive works that are mimetic; 
that is, they imitate actions, things, and events that do or 
might occur. Examples of such workS are computer 
Simulations of processes or events, such as the workings of 
a nuclear power plant or the development of a tropical 


Storm. 


A sub-class might be described as poetic, and it is 
differentiated from the class of mimetic interactive works 
in terms of itS end cause. Poetry as an art haS as its’ end 
cause the pleasurable experience of emotion. Even such 
emotions as fear and grief, the experience of which would, 
in life, be unpleasant, may be aroused and treated by poetic 
works in such a way as to provide pleasurable experiences 
for the audiences of such works. The emotional experiences 
afforded by poetry derive from the c pathic connection of 


ence to the piece; that is, emotions are experienced 





y by the audience. The absence of real-world 
consequences allows the audience to indulge in emotional 
experiences without risk, and poetic forms provide various 


kinds of resolution or catharsis which return the audience 


to a balanced emotional state. 
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Poetic interactive works have as their end cause to 
entertain, engage, or please the humans who play with then, 
inviting the user to partake of the vicarious experience of 
emotion and to delight in the imitation. The sub-class thus 
excludes mimetic works which are intended primarily to 
assist the human user by performing such tasks as 
computations, analyses, or instruction. Examples of poetic 
interactive works include video games such as "Space 
Invaders” or “Asteroids,” maze games like "Pac Man," 
adventure games such as “Zork,” and recreational simulations 
like "Poll Position." Some programs with the ostensible 
aqoal of educating the usSer also fall into the Bees class, 
as their primary value to the user is the pleasure of 
interacting with them. As Aristotle observed, man's delight 
in imitation derives in large part from the pleasure of 
a a principle which often eludes those who design 


both video games and instructional computer programs. 


Those poetic interactive works which imitate agents and 
their actions (as do most adventure games) and those which 


} user aS an agent in the action (as do most video 






recreational simulations) are composed of elements 


which “correspond to Aristotle's six elements of the drama 


12 
Aristotle, Poetics, p. 227. 
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(see Figure 1), An analysis of such works in terms of 
Aristotelean dramatic theory will suggest extensions of that 
theory in areas where poetic interactive works and drama 
diverge. That analysis will also yield a set of formal and 
Structural criteria which will then be used to design a 


Structural model of the IF system. 
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ELEMENT 


The whole action being 
imitated. The outcome 
of the action will be 

the same in each per- 

formance. 


CHARACTER 


Bundles. of predisposi- 
tions and traits, 
inferred from agents’ 
patterns of choice. 


Inferred internal 
processes leading to 
choice: coqnition, 
emotion, and reason. 


THOUGRT 


The selection and 
arrangement of words; 
the use of language. 


DICTION 


Everything that is 
heard. 
| SeECTACLE Everything that is 
seen. 


es. 
Be 


tg 
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IN POETIC 
INTERACTIVE WORKS 


The whole action, which 
is interactively shaped 
by both system and user. 
The outcome may vary. 
with each interactive 
session. 


The same as in arama, 

but ancludinda the uper 
as well as factitious 

aqents. 


Tne same as in drama, 
but includang the user. 


The selection and 
arrangement of discursive 
sians, including visuai:, 
auditory, and other non- 
verbal sians, when used 
linguistically. 


' Pigure 1-1: Corresponding Elements of 


Qualitative Structure 
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2. Functional Requirements of the IF System 

The IF system is intended to allow a human user to enter 
an imaginary world and to move through it as an active 
character, participating in an experience that is dramatic 
in nature. Contributions of persons with diverse 
theoretical and technical skills will be required to bring 
the IF system into existence. The system will necessarily 
be built by pieces, as essential techniques and tools are 
developed in many fields. The remainder of this section 
will be devoted to a discussion of the Functional 


requirements of the IF system (see Figure 2). 


2.1 Create a World 

The IF system nust provide a means for the fantasy world 
itself, the context of the interactive experience, to come 
into being. The physical characteristics, inhabitants, 
distinct environments, and laws of that world must be 
conceived and then represented in some manner to the user of 
the system. The world also includes some relevant history, 
which must be available to the system itself in order to 
orchestrate new events and dole out expository material to 


the cters (including the user-character). 





The conception of fantasy worlds is and will almost 
certainly remain a task for humans and not for machines. 


Playwrights and novelists perform Similar acts of creation 


when they define the worlds in which the actions they 
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Fig ‘Ls Overview of an Interactive Fantasy System 
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represent take place. The author of an interactive fantasy 
world, however, must cope with the additional requirements 
of the interaction. In a play or story, the author need 
only create those portions of the world which will be used 
by his characters. The author of an interactive environment 
must create a world that will support choices and actions on 
the part of the user-character which the author may not have 
foreseen. If the user is to have the widest degree of 
freedom in the interaction, the author must ultimately 
specify laws by which new environments and characters can be 
created, as conceiving of (and storing) representations of 
all possible phenomena in an imaginary world is an 


impossibility. 


The representation of the fantasy world to the user may 
take several forms. In many computer~based adventure games, 
the worle is represented in textual form on a screen. In 
other computer games, graphics, animation, and sound are 
used to represent the characters and the environment. The 


first-person requirement of the proposed system suggests 












that ultimate representation should have first-person 
sensa ‘Sharacteristics; that is, that it should occur in 
three~dimensional space and should invoke all the senses of 
the user. 


Whatever modes of representation exist in the interface 
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environment, a further functional requirement of the system 
will be: to provide the means for translating the author's 
description of the world into a representation of that world 
for hie user. Currently, it is the work of humans to 
interpret that descrition and then to prepare materials in a 
variety of media to represent it, much like the creation of 
sets and costumes for a film or a stags 
2.2 Make the Experience Interactive 

The interactive requirement means that the system and 
the user must be able to mutually and reciprocally influence 
the action. The "action” is composed of the events in the 
story that is unfolcing during the interactive experience. 
The implication, then, is that the interaction must be 
manifest on the level of plot. The first-person requirement 
further implies that that interaction. must occur in "real 
time;" that is, there may not be a perceptible delay for the 
processing of interaction (such a delay would constitute a 


destructive interruption of first-person experience). 


Dramatic form employs enactment as itS manner of 






therefore, the system should ultimately require 


Lice (i.e., non-enacted) inputs from the user. In 


13 


A description of a multi-media interface environment is 
provided in Laurel and Hulteen, "Simulation of an 
Interactive Fantasy System,” p. l. 
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concession to the limitations of current technology, 
however, a principle can be expressed that will maximize the 
ability. of the interactive design to meet the dramatic 
requirement: because interaction must occur in the context 
of the fantasy itself, it follows that all communications 
between the system ang the user occur in the modes in which 
the fantasy is represented. (This principle applies to 
traditional works as well: a novel, for instance, exists in 
a textual "mode," and its user interacts with it by 
reading.) If the fantasy world is represented in graphical 
Gisplays and animation, for instance, the system may not 
require the user to interact in a textual mode. The medium 
of communication is the fantasy itself. If the world of the 
fantasy includes speech from the user, then the system must 
employ speech recognition and natural language processing 


techniques. 


The plot, the selection (or creation) and arrangement 
(or order) of incidents in the experience, is successively 
formulated as action flows through the system. The 


re function of the system necessitates the design 





‘ures for communication between the user and the 
system that can elicit and interpret the kind of information 
which is required for the synthesis of the plot. Because 


the informational criteria are essentially dramatic in 


nature, the interactive and dramatic functions of the system 
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are closely related. 
= 

2.3 Make the Experience Dramatic 
2.3.1 Access to the World 

The interactive, real-time synthesis of plot constitutes 
the central dramatic function of the system. To accomplish 
that funtion, the system requires access to all the elements 
of the fantasy world, as created by the author. That 


information describes the dramatic potential and delimits 


the range of possible events and actions within the plot. 


2.3.2 Sensing the User 

Because the choices and actions of the user-character 
provide materials from which the plot is created, they can 
be seen as contributions to material causality in the 
system. A functional requirement of the system, then, is to 
be able to receive and formulate (give dramatic form to) 
those contributions. Material contributions on the levels 
of spectacle, music, and diction must be sensed directly 
(with the use of such techniques as body tracking and speech 


recogn; m). User contribuitions that are of the nature of 






| character must be inferred by some intelligence 


‘system itself. 
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2.3.3 System Knowledge and Inferential Capabilities 

The. system must have a way to know what is going on with 
the usee, the other characters, and the environment. It must 
create an internal representation of what has happened, what 
is happening, and what is likely to happen. In the latter 
case, the system must have a way to predict possible courses 
that Future events will take, and to assign a probability to 


each predicted course. 


The system must be able to interpret its knowledge of 
the past, present, and future course of events in a variety 
of ways. In order to construct a dramatic plot, it must be 
able to formulate and dynamically revise a description of 
the central action of the plot; that is, the whole action of 
which the events in the experience are parts. Further, it 
must be able to construct a logical hierarchy of actions and 


to trace the causal relations among them. 


2.3.4 Dramatic Expertise of the System 
In the creation of the plot, the system must be able to 
make several kinds of predictions and decisions. In order 


and maintain dramatic structure, the system must 






predict the probable shape of the whole action. 
In this way, the system provides for itself an ad hoc 


version of the finished piece which can contribute to formal 


causality. 
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The system must be able determine what should happen 


ees: 
her, 
Lae 


next. g,incidents generated by the system must conform to a 





terms of the action being represented (material criteria), 
in terms of the evolving dramatic structure (formal 
criteria), and in terms of their probable effects on the 


course of the action (causal criteria). 


The system must know the nature of and be able to 
produce beginnings and endings. That task requires the 
afore-mentioned ability to identify the central action of 
the piece. Another dimension of the task is the ability to 
Stimulate the user's cooperation in creating a beginning or 


an end for the experience. 


2.3.5 Creating and Animating Dramatic Characters 

Except in those fantasy worlds where there is only one 
character (portrayed by the user), the system is responsible 
for creating and animating other dramatic characters. Some 
characters may be explicitly required by the fantasy world 


he world of Star Trek, for example, always includes 







Spock. " The possibility for creating other 
8S may also be allowed by the fantasy world, and in 
those ¢ases, the system must be able to create’ them 
according to the laws of the worid and the exigencies of the 


particular action unfolding in the interactive fantasy 


experience. 
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Characters are imitations of agents, and are 


qgiffe ted from one another by their unique traits and 





predis ions. The system must represent those traits and 





predispositions for each character in actionable ways. 
Characters are also defined in terms of their goals and 
states. The system must be able to assess and influence the 
goals of each character and to monitor the changes in the 
various states of each character as the action proceeds. 
Por each character, the system must also maintain 
information about his relationships with the other 
characters (including the user-character) and the state of 


his knowledge about the world. 


i VRE y- 


Characters in the system may have varying degrees of 
intelligence, or self-motivation. They may be animated by 
the system either as puppets, or as self-directed agents 
whose actions are relatively autonomous. Given traits, 
goals, and predispositions by the author, for instance, 
characters may be most effectively animated by giving them 


the intelligence. to produce speech and make choices in 
af i oe wi ¥ i 








of situations. Examples of such “intelligent, 
eacters" exist in some artificial intelligence 


prog: Other researchers are attempting to create 





14 
See, for example, Meehan, "The Metanovel,” and Joseph 
Weizenbaum, Computer Power and Human Reason: From Judgment 
to Calculation (San Francisco: W. H. Freeman, 1976), 
Introduction and Chapter I. 
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pesonalities who are capable of generating their 


cal images and representing their actions through 





2.3.6 Maintaining Formal Control of the User-Character 

An important function of the system is to prevent the 
user-character from introducing new potential into the 
fantasy world; materials that are not represented in the 
fantasy world or admissible according to its laws cannot be 
handled by the system. Hamlet may not draw a laser weapon: 
a system operating on the knowledge of Hamlet's world would 


not Know what a laser weapon is. 


Vs 


To prevent the breakdown of the system or the necessity 
for second-person communications, the system must employ the 
plot to influence the internal probability structure of the 
user. The system must be able to create, through the plot, 
lines of probability that will draw the user away from 
"fatal" mistakes. While it cannot be asked to prevent acts 
of perversity or vandalism on the part of the user, and even 
adlingness on the part of the user to cooperate 


. 


m can be assumed, the system is solely 









15 
Research projects involving self-motivated, self- 
representing characters are being conducted by Ann Marion, 
Susan Brennan, and others at the Atari Sunnyvale Research 
Laboratory; reports forthcoming. 
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for guiding the action in a dramatically 








In other words, the system must assume ultimate 
responsibility for playwriting. As the master of the plot, 
the system has control of the form the experience may take. 
This formal control is necessary in order to guarantee that 
the structure of the piece is dramatic. It is also necessary 
in order to maximize the pleasure of the user. Theatrical 
improvisations of the classroom variety are seldom 
satisfying because the actor must perform two . different 
Functions at once, thus giving his full concentration te 
neither. He cannot lose himself in the experience of 
creating a character because he must keep an eye on the 
development of the plot; he cannot enjoy the act of 
playwriting because he is constantly distracted by the 
requirements of acting. By assuming formal control of the 
experience, the system frees the user to immerse himself 


fully in the experience of his character. 







and Focus of Future Work 

pers in this series will treat, first, the 
©f dramatic theory that are required to 
accommodate interactive structure, and second, the Gesign of 
the Playwright, the portion of the system in which dramatic 


expertise resides. The objective of the series is to arrive 


at a structural model of the Playwright and to delineate 
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relations between the Playwright and the rest of the system. 





on of the IF system in operation, using a human in 


the of Playwright, is planned by ASRL, to be used 





variously as.an aid in modeling the Playwright, a means for 
refining the overall system design, and a way to create a 
sensible example of the kind of experience that the IF 


system is intended to enable. 








